Sunday, May 12, 2013
Ive been pondering this one for a while and been meaning to put something out there about it.
During some of the architecture discussions I've had with colleagues some of the examples that are often put out there in terms of a public API are google and twitter. They are often described as things like "internet scale", "open standards based", "easy to use" and a whole bunch of other good things.
One of the things which I always feel that is a little bit ignored is that these companies are very different in terms of their market forces to some of the companies I have worked with in the past. If you are twitter for example its very easy to say this is my API and you will consume it using my rules. If twitter want to change their API or retire an older version of it then the client either upgrades or stops working.
If you are a service provider like one of these guys then you have all of the power and its really easy to produce an API and make everyone do it the way that suits you. I would describe this as a "provider driven API".
A lot of the companies I have worked with are different. They work with partners and customers who have a lot of power and the balance of power in their interactions and relationship is either finely balanced or in favour of the service consumer. In these cases we often end up with different scenarios where you may have an API, but you also end up having to build adapters or gateways to your API so that different consumers can work with it. To give an example of this we planned to produced an API but we found that one of our biggest consumers would account for 70% of its usage and they could only support a SOAP web service API with basic security where as another consumer of it would be REST based. In this scenario you may produce your "preferred API" and then produce adapters to convert different consumers to this API. The power balance means that you cant necessarily influence the consumer to change the way they would talk to your API because they would use a competitor who would make it easy and cheaper for them. Thats just the nature of business. In these cases I find that its important to try to use good integration practices and to keep the adapters as simple as possible and to reuse the core services in your preferred API. This will keep your total cost of ownership lower. I also often suggest to the business that they look for ways to incentivise partners to use our preferred API as this brings our costs down so we should look to share that saving for the longer term benefits it would bring. We know this isnt always possible though. In this case I would describe your API strategy as being a "consumer driven API".
I think its important to recognise that there are differences between these approaches, and that while google, facebook, twitter or Windows Azures's API might be excellent and very easy to use, they may be in a different business paradigm to your company and while there are definitely good techniques and practices to aim for they may not be the best comparison to your own situation.
This subject is not something I have really researched too much online to see if there is much in the way of discussion on this subject but im bored sitting waiting for a flight in miami airport and thought id get this one off my chest!
Id love to hear what other people think, is this a valid problem space that lots of us face, and what kind of challenges do you have in your domain. Also do you think the terms "Consumer driven API" and "Provider driven API" are right. Ive come across the "Client driven contracts" pattern but I feel that this is only part of the challenge.
From my own experiences the one good practice I would recommend in the "consumer driven API" space is that you should try to avoid developing whole new API's for all of the different consumers. Create your main API and then develop gateways to it. The gateways should be as simple as possible and do little more than mapping and composing the data from the main API into these newer formats and dealing with the differences in things like security. Dont go replicating all of the logic if possible.
Saturday, May 11, 2013
I've been thinking for a while about the way in which resource based architectures and messaging based architectures interact with each other. Resource based architectures have increased in popularity a lot over recent years and I have read a number of articles in the community but I've always felt there was a gap and lack of discussion and guidance over the integration between resource based architectures and message based architectures. I have had a couple of chats with colleagues who's opinion I value very highly and they felt the same about this subject as I do so I thought I'd write this article and see where we end up at the end of my brain bump on this subject and hopefully prompt some discussion on this subject.
So let's start with the typical example of a RESTful service. In most of the examples you see in articles we have an application that contains some resources and then a REST API is developed to provide access to these resources using all of the common techniques used when putting together a REST API. At this point I am completely happy. REST is a great way to provide an API for your applications resources. "REST is good for producing an interoperable API".
If you're developing applications in a client/server style architecture where you may have a website which calls this REST API then you probably never need to worry anymore about this problem, however where I start to struggle with things is in the real world of integration. When we start to develop real solutions which are more than just a website calling my API and we have things like aggregated services or complex business processes or the many other integration scenarios to think about. Let's take the example of a service which wants to aggregate two or more services together in a publish/subscribe style pattern. In the below diagram I've shown how you have two backend services exposing REST API's and a client who wants to make a call with a RESTful GET.
There are a few conceptual challenges here, firstly when you are talking about resources, the resource that the customer is thinking about isn't necessarily the same as the resources that each partner is thinking about. The message broker could be aggregating the two partners resources and producing some new kind of resource. Other challenges may include the addressability of the resource. Would the address send by the client make sense when it was sent to the partners. Probably not!
In the above example it could be possible to write custom code to deliver this aggregator service but what about a more complex problem? Take the diagram below.
In this example we have a more complex example where there are multiple customers and more partners. Now like in most real world scenarios you have more than one protocol going on. Now you're moving away from custom code, it would be a good to consider some of the common messaging solutions such as BizTalk, NServiceBus, RabbitMQ, or the many other vendors who are available.
The key thing when you start working with an integration broker is that they work based on a message based architecture and a key underlying principle of that is a message which is self-describing. A message based architecture has the benefit that because everything is contained within the message it should be possible to route then transform the message and send it over any protocol. Hopefully this begins to show how important the translation from a REST à Messaging and Messaging à REST can be.
In the new version of BizTalk 2013 they have introduced a new REST adapter and the BizTalk product team have attempted to address this challenge about how to convert things like a URI with the GET verb to a message which can then be self-describing and used in a messaging system. The REST adapters then let you connect applications to your integration broker and provide whatever mapping and routing your scenario may need.
In summary I think I'm saying the following:
- REST is an easy to use and interoperable way to develop application interfaces
- There is a difference between an application interface and Enterprise Application Integration
- Messaging systems use messaging architectures and are used to develop real world EAI and ESB solutions
- BizTalk has some cool new REST adapters which let you convert between these architecture styles
- We should be able to build even more powerful integration solutions than we have in previous versions
As I mentioned at the start of the article, I haven't seen much discussion in the community about this translation from Resource-based to Message-based approaches. If you know of any good articles or have any opinions around good practices or techniques in this area I'd be very interested to read them.
As a side note if anyone is interested in practical examples of using the BizTalk REST adapter check the below articles from fellow Integration MVP's Richard Seroter and Steef-Jan Wiggers:
Sunday, May 5, 2013
Ive been trying to get the sample for the BizTalk Map unit testing scenario working. I kept getting the following error:
"Transformation Failure"
This was happening everytime I executed the test even though you could test the map fine in the Visual Studio IDE.
While troubleshooting this I also tried using code which would execute the map in C# outside of the BizTalk Unit Testing feature based on Tommaso's article. When I did this I was getting the following errors:
- The type initializer for 'Microsoft.BizTalk.ScalableTransformation.BTSXslTransform' threw an exception.
- The type initializer for 'Microsoft.BizTalk.CommonSettings.CBizTalkSettingsLookup' threw an exception.
- Catastrophic failure (Exception from HRESULT: 0x8000FFFF (E_UNEXPECTED))
When I ran Tomasso's application it was able to execute the map successfully which lead me to believe that it must be something to do with the fact the code would be running inside of the MsTest execution environment.
Eventually I found a solution to this. I added a test settings file to my solution and referenced this in the TEST menu so it was the active test settings. When I did this running the test with the BizTalk Unit Testing feature and running the test using C# to execute the map both worked fine. I didnt need to modify any settings in the test settings file, I just left them as the defaults.
Im not quite sure of the exact cause of the problem but this fixed things for me. Its a shame the error messages in this case made things really painful to workout the root cause of the error.
Monday, April 29, 2013
Ive been interested in exploring the opportunities around the Fakes Framework for testing BizTalk solutions for a while now, and until recently its been sitting as one of the many blog articles I have intended to write but haven't got around to. This week we have started looking at a BizTalk 2013 project and this presented an excellent opportunity to explore this idea.
I decided to look at the common problem people have faced in the past when you want to unit test some code and it has a dependency on the underlying BizTalk assemblies and they were difficult to mock out with one of the common mock frameworks. One of the most common examples was when you write a helper class for an orchestration and the method takes an XLANGMessage object as an input parameter. This is quite a common pattern that people have struggles with. Often the way people would handle this was to write a method that takes XLANGMessage and in that method extract the message body then call a 2nd method which did the logic required. This approach meant you could easily unit test the 2nd method and the first method people would often skip unit testing.
Although this wasn't ideal, it often worked and we would just accept this method would be tested as part of the BizUnit tests of the entire solution.
Let's take a look at a sample method that we may be trying to test.
You can see that we want to extract the message body from the BizTalk message and then return it as a string. The big problem with unit testing this method is the dependency on the XLANGMessage object and how difficult it could be to correctly construct this object to pass into the test.
Since we now have Visual Studio 2012 available this brings in the Fakes Framework and we have a few new things available to us which will make this easier. First we need to look at our references to a couple of the underlying BizTalk assemblies and create Fakes for them. To do this right click on the references assembly and click Add Fake Assembly
Once we have a Fakes assembly we can then create Shims and Stubs of the various objects which are within this assembly. Im not going to go too much into the detail behind shims and stubs, but will provide some links to reference articles later.
We can now write a unit test where we can fake the reference to the underlying BizTalk assemblies and make them much easier to work with and allow us to focus on the method we want to test. The below code snippet shows how we will create our test. It will call a helper method which will create the shim of the XLANGMessage and then pass it into our method to test.
You can see in the above method that our test becomes very simple while still using the XLANGMessage or BTXMessage which derives from it. One key point to note in this method is the use of the ShimContext and its disposal which ensure the shim framework is created and cleaned up after our tests have used it.
Now lets look at the method which creates our message.
You can see this method is where the fakes framework magic is happening. We firstly create a stub of the XLANGPart object and assign our string to be its stream that is returned when we call the RetrieveAsType method. We then need to create stubs of a couple of other classes which are needed to create the stub of the BTXMessage. On this message we set the return for the intexer property to be our XLANGPart which we created earlier.
Finally we create a shim of the BTSMessage and return it.
This method for creating a fake XLANGMessage is probably quite reusable for most of your unit testing scenarios as we tend to usually interact with the message body in a helper class but not that much else.
Conclusion
Hopefully this example has shown you how easy it can be to use the Fakes framework to test code which you may have previously thought was untestable. This happens a lot when you have dependancies on the underlying BizTalk assemblies in your unit tests.
I think there could be quite a few other opportunities where this framework could be used to complement our existing BizTalk testing practices and I plan to explore these over the coming weeks.
The code for this sample is available on the following link:
https://s3.amazonaws.com/CSCBlogSamples/ShimExample.zip
References
I mentioned earlier Id point out a few other sources if your interested in the fakes framework and want to find out more. Id probably start with the following:
Ive been bitten by this one a couple of times recently and want to remind myself for next time!
MsBuild doesnt shutdown after its been ran by default (performance improvements I expect). For integration development this is often a problem with bits being locked etc.
This can be turned off by adding the following environment variable:
Name: MSBUILDDISABLENODEREUSE
Value = 1
Friday, April 26, 2013
I work with a number of customers who use Windows Azure and are increasing their use of Windows Azure due to the significant success they are having. One of the things where I feel customers are yet to create mature processes is in their understanding of how to best leverage the billing side of cloud services. You may not be familiar with a Windows Azure Enterprise Agreement if you only use Windows Azure for personal use or small business. Basically an Enterprise Agreement is a way customers can create an agreement allowing them to have an account they can add money to and then they can have multiple Windows Azure Accounts and subscriptions which will be related to this agreement and they will be billed against this central pot of money rather than having to have lots of credit card payments which most people are probably familiar with.
One of the big benefits for larger customers is that the commitment of cost and economy of scale means they get lower prices because they will be bigger users so if you're a corporate customer then an EA is definitely something you should look at.
If you have a Windows Azure Enterprise Agreement then you get access to another portal at the address http://ea.windowsazure.com. This portal allows you to manage the agreement and to access the billing side of things.
One of the things I was particularly interested in is the underlying billing data which you can download from the EA portal.
I have created a spreadsheet which you can paste your billing data into and then there are some worksheets which will give you some interesting views of that data. (Download the spreadsheet and refer to the instructions worksheet to see what you need to do). Note that on the detailed data worksheet I've added some additional columns which do some basic calculations to format the data in a way its easier to analyse.
If you want to see an example of the spreadsheet with some sample data please check it out on the following link:
http://tinyurl.com/d5v5799
The views you can see are as follows:
Agreement Overview
On the agreement overview worksheet there are two simple views. There is one which shows which accounts are showing up in the billing data and one which shows which subscriptions are in the billing data. This can help you to workout if any accounts or subscriptions are no longer being used.
Cost Overview
The billing data you have downloaded will cover a specific date range. The Cost Overview worksheet will give you a view of the overall cost for each subscription and account for this billing period.
This is a good view for your purchasing and accounts team so they can workout at a high level the cross charging that may be required.
Total EA Costs by Month
This gives me a high level total cost spent on Azure each month.
Costs by Month and Subscription
This is a good view for the Bill Payer to work out how much each subscription is paying each month. This can help me with cost allocation within the organisation.
Costs by Month and Service
This worksheet will give you an overview of costs by month and type of service. You will be able to answer things like "How much did I spend on Compute and SQL in April 2013". This view is one of the key areas where the cross over between bill payer and architect comes into play. As an architect I can check if the running costs for SQL are in line with my expectations, or spot if a component may have a defect if its costs seem higher than expected.
Component Costs
This worksheet gives you a way to workout the cost per month and year at a component level. This is a good way to spot components with an unexpectedly high cost which may need investigating.
Component Overview
The component overview worksheet gives me a high level view of the components under each account and subscription. I can easily see what type of services they are and where they might be deployed. As an architect you often design an architecture but you know that as a project progresses it is likely to change. By monitoring the billing data I can spot unexpected components creeping into the development and testing environments which may not have been communicated by the development team.
This is also a good holistic view of my component estate in Azure.
Component Costs to Review
This view is very similar to the component costs except that we have used a filter of the overall cost over the charge period to highlight components where we may want to validate the costs to ensure there isn't something to investigate.
Regions Used
This view in the Regions Used worksheet gives me easy access to look at the components in my agreement and what region they are deployed to. As an architect I may have requirements which mean my development team should not be putting components in specific geographical regions and I may use this data as a way of checking overtime that any rules like these are being adhered to.
Overview
I believe over time the Azure Enterprise Portal will include a lot of analysis features because this is really important for accounts people but also architects too. Now a days we need to think a lot more about running costs for our solutions than we did in the past. Access and analysis of this data is very important.
Tuesday, April 23, 2013
If you are installing BizTalk 2013 and get the error "
Internal Error 2761"
Just check you remembered to right click and Run As Administrator on the BizTalk setup.exe
I was recently working with a customer around the initiation of a new project and we had to make a decision about wether we should choose BizTalk 2010 or BizTalk 2013 for this project.
Obviously being a keen techy I was actively encouraging the choice of BizTalk 2013 but as any responsible architect knows its important to follow a structured decision making process to evaluate the pro's and con's of the different options and make a sensible decision for your organisation and to also have justification for that decision.
We use a lightweight architecture decision framework loosly based on the excellent Applied Architectures on the Microsoft Platform books framework.
The powerpoint slides on the link below shows you some of the things we considered when we made that decision which might help you.
Id love to hear any thoughts from people on this approach good or bad
Monday, April 22, 2013
This week we came across the below error on in our project where we are using Windows Azure Service Bus Brokered Messaging.
Microsoft.ServiceBus.Messaging.MessagingCommunicationException: Error during communication with Service Bus. Check the connection information, then retry. ---> System.ServiceModel.CommunicationObjectFaultedException: Internal Server Error: The server did not provide a meaningful reply; this might be caused by a premature session shutdown..TrackingId:4990b52f-7588-4460-81eb-cbef7b103c11, Timestamp:4/22/2013 2:32:44 PM
We initially thought this was a networking issue, we had been having separate issues with our developer machines and networking and initially thought it was related to that. However on closer inspection you can see there is a tracking id in the error message which gives you a good clue you probably managed to make a call to the service bus.
We found the root cause of our problem was that we had put some logging in the code to log the message body before sending the message and when we had read the stream which contained the body and written it to the log file we hadnt set the position back to the start of the stream before sending the message.
Ive seen a few forum posts about this error which seemed to be related to other things but didnt see anyone have the above symptoms so sharing it incase it helps anyone else
Sunday, April 21, 2013
I came across a something the other day with WCF service I was hosting in IIS which was configured to use the relay bindings to listen to the Windows Azure Service Bus.
I had made an error in the configuration file and it popped up with the below error in the browser.
Server Error in '/Acme.Azure.ServiceBus.Connect' Application.
Description: An error occurred during the processing of a configuration file required to service this request. Please review the specific error details below and modify your configuration file appropriately.
Parser Error Message: Invalid element in configuration. The extension name 'transportClientEndpointBehavior' is not registered in the collection at system.serviceModel/extensions/behaviorExtensions.
Source Error:
Line 159: <endpointBehaviors> Line 160: <behavior name="BrokeredMessageService-Ws"> Line 161: <transportClientEndpointBehavior credentialType="SharedSecret"> Line 162: <clientCredentials> Line 163: <sharedSecret issuerName="owner" issuerSecret="*** I have hidden the key but its displayed here*** |
As you can see from the bottom of the error message this displayed the shared secret key (obviously I have hidden it in this post).
Now we all know how from time to time a configuration error can happen, and although its unlikely if it did happen you dont want your key comprimised.
Fortunately there are a couple of ways you can probably handle this.
1. You can add a custom error page as described in the below link: (thanks to Jayu Katti on the Service Bus Team for this one)
2. You can encrypt parts of the configuration file as discussed in the following PNP article. I havent tried this one yet but the assumption is if you can encrypt the appSettings and connectionStrings sections you can probably encrypt the whole of the system.servicemodel section
Anyway hope this helps a few people
Thursday, March 28, 2013
Recently I have been working with a customer where we wanted to migrate some source code to TFS Services for source control but we had an existing Build Server setup running on-premise with Cruise Control. We did not want to change this over to any other continuous integration server so we were hoping to just point to our new TFS server in the cloud and it would all be straightforward. Unfortunately it wasn't this easy.
The core of our problem is that the CCnet plug in for Team Foundation Server (the vsts source control plugin) is implemented as a wrapper for TF.exe. TF.exe seems to only work with TFS Services when you have logged in with your Windows Live ID credentials. There are a few articles out there about automation of TFS Services but to be honest at this stage there seems to be a lot of people struggling with how to automate properly and particularly around the different security options. Also bottom line is that none of the articles I have seem discuss how to make TF.exe work with TFS Service without logging in via Windows Live ID.
Eventually I have managed to get this scenario working but it was quite painful so ill document it here to help anyone else who may be struggling. The approach I have taken refers back to the TFS Plugin on Codeplex for CCNet (http://tfsccnetplugin.codeplex.com/). This plugin doesn't seem to have been updated for ages so I'm not sure if it is still maintained now that CCnet has its own TFS provider. The key thing about the codeplex plugin though is that it uses the TFS assemblies rather than TF.exe so I'm going to modify that plugin to work with TFS Sevrice and the rest of this article will show you how to do it.
Before you start
Before we get into the details of this we will assume that you have just installed the latest version of Cruise Control.net. I have tested this with version 1.8.3 but I'm pretty sure it would work with earlier versions too.
Step 1: Getting your TFS Credentials
Before we get into the CCnet space, you will need your TFS Services credential set which will allow you to authenticate without Windows Live ID. To do this you need to use the TFS Credential Viewer Tool
The tool is available on the following page: http://blog.hinshelwood.com/tfs-service-credential-viewer/
The following video also shows you how to use the tool: http://youtu.be/Fkn6V0_zz28
Once you have used the tool to get your credentials you will have the following information:
Step 2: Modifying CCnet to work with .net 4
We will be writing a new plugin for CCnet to use TFS Service which will need to use the TFS objects which come with Visual Studio 2012, now you may be able to get away with writing this in older versions of Visual Studio but I was having issues with this so I have written the code in Visual Studio 2010 and targeted .net 4.0. Because Cruise Control is a .net 2.0 application by default I have modified it to run in the .net 4.0 runtime instead. If you need more info on this please refer to the following article: http://www.davidmoore.info/2010/12/17/running-net-2-runtime-applications-under-the-net-4-runtime/
To implement this change I have modified the following configuration files in the Cruise Control Server folder:
- ccnet.exe.config
- ccservice.exe.config
I have modified the runtime element to include the highlighted test telling .net 4 to use the legacy security policy.
<runtime>
<assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
<dependentAssembly>
<assemblyIdentity name="NetReflector" publicKeyToken="2f4dd8b32acbcd8e" culture="neutral" />
<bindingRedirect oldVersion="1.0.0.120" newVersion="1.1.2009.1214"/>
</dependentAssembly>
</assemblyBinding>
<NetFx40_LegacySecurityPolicy enabled="true"/>
</runtime>
I have also added the startup element to tell it to use .net 4 but to support .net 2.
<startup useLegacyV2RuntimeActivationPolicy="true">
<supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.0"/>
</startup>
Step 3: The CCNet Plugin
I have taken the code from the Codeplex TFS Plugin and modified it in two ways.
Firstly I have changed and parts where it used the AuthenticatedUser property of the sourceControl variable to use the AuthorizedUser property instead.
The 2nd change is to the TFS property where I have changed how the TeamFoundationServer reference is provided. It will use the SimpleWebTokenCredential so we can use the Identity we got credentials for in step 1 rather than an active directory based identity that you would have for an on-premise TFS instance. The below code snippet shows this:
var creds = new
TfsClientCredentials(new
SimpleWebTokenCredential(Username, Password));
var tfs = new
TfsTeamProjectCollection(new
Uri(this.Server), creds);
return tfs.TeamFoundationServer;
The code for this plug in is available at the following location:
https://s3.amazonaws.com/CSCBlogSamples/CCNetTfs.zip
One key point to node about the plugin is that the dll name needs to be ccnet.*.plugin.dll so that it is picked up by the reflection process used by CCnet. When I have compiled the assembly I simply copy it to the CCnet folder where the ccnet.exe file is.
Step 4: Modify the CCnet.config File
The ccnet.config file contains the definition of the project. I will need to modify the source control element to use my new plugin. The configuration for the plugin is very similar to the out of the box vsts source control plugin except that you don't need the executablePath property because we aren't using TF.exe. You would need to supply the username and password properties that you wouldn't usually supply on-premise (unless you were using a different account).
An example of this configuration is below (note I have highlighted the name of the new plugin):
<sourcecontrol type="tfsservice" autoGetSource="true" applyLabel="true">
<server>https://<yourTFSname>.visualstudio.com/DefaultCollection</server>
<project>$/<Your project path></project>
<cleanCopy>false</cleanCopy>
<deleteWorkspace>false</deleteWorkspace>
<workspace>TestWorkspace</workspace>
<workingDirectory>C:\TFS</workingDirectory>
<username>Account Service (<yourTFSname>)</username>
<password><My Password></password>
</sourcecontrol>
You are now ready to restart your ccnet windows service and the project should pick up your new plugin.
Conclusion
Although it was initially painful to figure out how to do this hopefully you can see that Cruise Control can be used to connect to TFS Service to build your code.
Sunday, March 24, 2013
Just wanted to put a note out for anyone who gets caught out like I did on this one.
I was trying to create an azure VM and kept getting the below error message and Azure wouldnt even create my VM.
"The word or phrase you selected is not allowed"
I tried different combinations of all of the settings but with no luck!
It turns out that apparently there is a list of naughty words and common phishing terms which you are not allowed to use. The below forum posts talks about this but in relation to web and worker roles it also seems to apply to virtual machines.
Just be a little careful over what you call things as you might not realise that your name contains one of these phrases at first glance and there is no list publicly available to check against.
Friday, March 15, 2013
Recently I wrote an article for UK MSDN Flash about using Windows Azure Service Bus Queues & Topics
Ive had some great feedback on this article from friends! Check it out below
Tuesday, March 5, 2013
I was having a chat with one of the guys in the office today and was wondering about the considerations you would make when deciding between a web/worker role and a VM in Azure.
To give an example we have a solution which runs some web services and ASP.net pages in a web role component which has 2 small instances and also some background processes which run in a worker role which has 2 small instances.
This configuration gives us enough scalability at this stage and gives us a basic high availability scenario.
The costs we would expect to see for this set up would be around $350 per month based on the Azure list prices (we actually have an enterprise agreement but for the sake of a simple example lets just go with the list prices).
If you consider this to a traditional model you might run these services on the same server if you werent in the cloud. EG host the web services in IIS and the worker role as a windows service. So we could potentially get away with just 2 servers in the above scenario because our 4 roles have spare capacity. If we transfer that to the cloud and considering hosting in IAAS instead we could use VM Roles and get away with 2 VM's and host like I described earlier as if they were on premise.
Now obviously VM is in preview at present so the costing is likely to change but the difference could be significant if the prices are similar between the roles when VM goes into general availability.
So coming back to my original thoughts which were along the lines of what considerations you would make and what would be the tipping point between deciding to choose between a VM role co-hosting multiple of your services versus a Web/Worker Role. I guess one of the most obvious considerations would be the cost of managing the server myself and patching etc. I think one of the others would be the auto-scaling benefits of the web/worker role which is probably a lot simpler than in the VM role in terms of application deployment.
Im quite interested to see what other people think in this area, particularly based on real-world examples
Saturday, February 16, 2013
Following on from previous samples I wanted to talk about how to implement a scatter gather pattern using the AppFx.ServiceBus framework. In this sample we want to achieve the following:
- Send a message from a client application to a Service Bus topic
- Use the topic filters to route this client to two subscriptions
- Have a service listening to each subscription which will handle the message and return a response
- Have the client wait for multiple responses to come back and return them as a collection.
Hopefully that is straight forward so let's walk through this sample.
Accessing the Sample
On the codeplex site you can download the samples. In the samples folder you need to go to the folder: Sample5-ScatterGather
This folder contains all of the code for the scatter gather sample.
Configuring the Azure Service Bus
To begin with we will need to create the following in the Windows Azure Service Bus
- A response queue which response messages back to the client will be published to. This queue will need to have session enabled
- A topic which the client will publish messages to
- 2 x subscriptions, 1 for each client
- A rule for each subscription using 1=1 so that the message is routed to all subscriptions
The response queue will be called: sample5-scattergather-response
The topic will be called: sample5-scattergather
The subscriptions will be called:
- sample5-scattergather/Subscriptions/server1
- sample5-scattergather/Subscriptions/server2
Note that in the samples folder there is an xml file containing the setup for all queues and topics used in these samples which you can import into your namespace using the Service Bus Explorer tool.
The message contract
The message contract follows the usual standard from previous samples. We will use an xsd schema to define out messages and then generate classes to represent these. In the sample the client and server will use this object but as data goes over the wire so to speak it will be in a serialized format which can be either xml or JSON depending on what your specify.
The contract looks like the below picture. You can see that again we are using the get customer sample message.
Configuring the Client
Moving on to the client where we have a simple windows application we will need to begin by setting up the configuration. In the same way as previously we will use the AppFx.ServiceBus configuration section. Firstly I will add my connection string for my service bus namespace like in the below picture.
After the section is declared and also the connection string I will add my AppFx.ServiceBus section and configure it to talk to the Topic in the Windows Azure Service Bus. See the below picture:
You can see that above I have added my client and specified the queue/topic to send the message to and also my response queue. Ive also specified my serialization format.
Client Side Code
Next in the client side code I need to use the SessionScatterGatherMessagingClient class. When I have constructed this class I have used Generics to tell it what type of messages I will be dealing with. I have also specified a client name so it knows what configuration to pick up from the config file (note you cant quite see that in the pic).
When I come to send the message there are a couple of key settings to be aware of. Firstly in the constructor you can override the default timeout if you want to. You can also in the SendScatterGather method specify the minimum number of responses you want to get back. This means that the messaging client class will wait for 30 seconds for responses to come through but if the response queue is empty before 30 seconds but the messaging client has already got your minimum number of responses back then it will return to the caller with what it has regardless.
Hopefully you can see that the client code is pretty simple.
Configuring the Server
On the server side we will take a similar approach to the previous samples. In the ..\Library folder (in relation to the solution file) there is three folders. One has the dll's for the client side of the framework which the client will reference, but the other two contain instances of the AppFx.ServiceBus console host application which will be used to listen on the service Bus.
In the server project in Visual Studio we have two configuration files:
- Server1.AppFx.ServiceBus.Hosts.Console.exe
- Server2.AppFx.ServiceBus.Hosts.Console.exe
These files contain the configuration for each of the hosts to connect to the service bus. The only real difference between then is the message entity they will listen to. Each host will listen on a different subscription. An example of this is below:
During the build process for this project I have modified the build using the OverrideBuild.targets file in the project so that these two configuration files and the server and contracts assemblies are copied to the right server folders so they can be picked up by each host. If you compile the projects and then check these folders you will see they are copied over.
The server Message Handler
The message handler for each server instance is exactly the same. It will simply return a response. An example of the code is below:
Running the sample
To run the sample, click F5 and the client application will open up. You will also need to go into the ..\library\AppFx.ServiceBus.Hosts.Console…. Folders and run the AppFx.ServiceBus.Hosts.Console.exe file. This will start up the servers listening on their subscriptions.
When you click send you should get a response shortly afterwards displaying that you have responses like the below:
You should also be able to check both server console windows and see that they have processed messages like the below:
If you look carefully at the text in the console window you will see that the message was received and handled by the message handler. You should see the same in both windows.
What about Errors?
It is possible that one or more of the listening services could return an error. Using the AppFx.ServiceBus framework it you have a pattern available to help you deal with this scenario. If any servers return an error then this is returned in the same way that errors were returned in the RPC samples. The key difference is that using the Scatter Gather messaging client class it will catch these errors and return them to you at the end of the call in the response object. You get the reply which contains a list of good responses and a list of error responses.
The calling client can then deal with these however it needs to.
Summary
Hopefully you can see this walk through makes it nice and easy to implement the scatter gather messaging pattern. I think in a future release we will also include another option with the scatter gather pattern so that the message client will raise an event each time an error or success response comes back as I can also see scenarios where that will be useful too.
Just released a new version of the AppFx.ServiceBus framework for Windows Azure Service Bus.
This includes:
- Performance optimisations
- Scatter Gather implementation changes
https://appfxservicebus.codeplex.com/releases/
Tuesday, February 5, 2013
In this article we will revisit the first sample about one way messaging and look at how the AppFx.ServiceBus framework implements error handling and retries. In this article we want to achieve the following:
- We will take the code from sample 1
- We will modify the message handler to throw errors.
- We will see that the message will be retried
- We will see that eventually the message will be processed successfully
Getting the Sample
The sample is available in the source code section on codeplex:
http://appfxservicebus.codeplex.com/releases
In the source code and samples download you will see a samples folder and you want the folder called Sample4-OneWayErrorHandling.
Im not going to go into the full details of the sample like we did in previous samples so I will assume you are familiar with sample 1 on the following link:
http://geekswithblogs.net/michaelstephenson/archive/2013/02/03/152012.aspx
In the rest of the article we will talk about how that sample has been modified.
Azure Setup
The setup of queues and topics to support this sample is exactly the same as for sample 1 and in fact it uses the same queues.
Modifying the message handler
In the message handler we are going to extend the logic. We will use the some context information from the MessageProcessorContext class. In that class there is an instance for each processed message which gives the message handler developer access to the Azure Service Bus Message and other things in an encapsulated way. From that class in our message handler we are interested in the MessageProcessorContext.Current.DeliveryCount property. This property tells us how many times the message has been attempted to be received from the Service Bus queue or subscription. In the error handler we will look to throw an error if the delivery count is less than three. The framework will reject this message and Azure Service Bus will make it available for collection again. The framework will then again try to process the message. In our logic the message handler will allow the message to be successfully processed once the delivery count is 3 or greater.
See below for an example of the message handler.
Running the sample
Rather than talking through the whole running of the sample, if you remember back to sample 1 you need to do the following:
- Compile the visual studio solution
- Start the AppFx.ServiceBus.Hosts.Console.exe application in the library folder
- Start the windows test application
- Click the button
When the button is clicked within a second or so your message will be received by the console application and you will see that in the red text indicates that the message was received and an error occurred. The message was rejected. This happens twice and then on the third attempt the message is processed successfully.
Other Considerations
Below are some other considerations you may want to think about when handling and throwing errors.
Azure Max Delivery Attempts
One thing to consider is that in the configuration or your Azure queue or subscription you can set a value for the maximum attempts for redelivery. This setting still comes into play when using the AppFx.ServiceBus framework. If the framework rejects a message 4 times and the Azure Queue max deliveries is set to 4 then Azure Service Bus will send the message to the Dead Letter queue.
AppFx Exception
In the framework there is an exception called AppFx.ServiceBus.Exceptions.ApplicationException. This exception allows you to throw an error in your message handler and to have the framework use the properties on the exception to allow you to control if a retry is attempted or not.
If an exception is thrown which is not the above type of exception then the framework will implement the behavior to keep log the error and keep retrying the message until the Azure Service Bus max delivery attempts kicks in and dead letters the message.
Summary
This sample should show you how the AppFx.ServiceBus framework deals with the retry and error handling scenarios you are most likely to need. This is one of the key areas we feel that AppFx.ServiceBus really makes life simpler for you.
Monday, February 4, 2013
In this article I will discuss extending the RPC sample from the previous article to include some error handling incase the server could not process your message. In this scenario we want to achieve the following:
- Setup Windows Azure Service Bus to support this sample
- Define a schema for our message and generate the .net types which represent it
- Create a message handler to handle the message
- Start the host to listen for messages
- Create a client application to send the message
- Have the server reject the message because of some error condition
- The client will get a response message via the response queue
- The framework will convert that response message to an exception and throw it for the client
Let's get on with things.
Setting up Azure Service Bus
In this sample we will use two queues. The first queue will be one which will be listened to by the server side host. This queue will be called Sample2-RPCRequest. You can create all of the appropriate queues for the sample by importing the Appfx-Servicebus-Samples_Entities.xml file which comes with the source code.
There is nothing particularly special about the Sample2-RPCRequest queue.
We will also create another queue called Sample2-RPCResponse. This queue is used when responses are sent from the server back to the client. One thing to note about this queue is that it has the "Requires Session" property set to true. This is so that the queue can receive the responses for multiple requests and have different listeners concurrently listening to the queue for their own response by using a session (note this is taken care of by the framework).
Creating your schema & message
When creating our message with AppFx.ServiceBus we prefer the technique of creating a schema to represent your message. This means we have a good contract for the message which we can then create classes in most programming languages to represent these schemas. This also allows us to serialize an object based on this schema to numerous formats such as XML or JSON.
One convention which we do use which is quite important is borrowed from BizTalk world where we use the combination of xml namespace + root element name to uniquely identify a message. The AppFx.ServiceBus framework will use this to be able to work out how to handle a message and how it can be serialized and deserialized.
In this example there are two messages (the request and the response), the message types are
http://contracts/v1.0#GetCustomerRequest
http://contracts/v1.0#GetCustomerResponse
Note: Please note the # character used to separate the namespace and root element name
In the sample we will begin in the Contracts c# project. Here we have created a schema in the schemas folder called contracts_v1_0.xsd and we have populated this with a simple definition of a type shown below:
<?xml version="1.0" encoding="utf-16"?>
<xs:schema
id ="Contracts_v1_0"
xmlns="http://contracts/v1.0"
elementFormDefault="qualified"
targetNamespace="http://contracts/v1.0"
xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="GetCustomerRequest">
<xs:complexType>
<xs:sequence>
<xs:element name="MessageID" type="xs:string" />
<xs:element name="CustomerId" type="xs:string" />
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="GetCustomerResponse">
<xs:complexType>
<xs:sequence>
<xs:element name="CorrelationId" type="xs:string" />
<xs:element name="CustomerId" type="xs:string" />
<xs:element name="CustomerName" type="xs:string" />
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:schema>
Now you would be able to give this schema to other parties who should be able to create classes from it and use them, but in our very simple demo we will next generate a class from this schema and keep it in the contracts assembly which will be shared either side of the queue just to keep it simple.
In the overridebuild.targets file you will see how we generate the class using svcutil which will produce a class in our contracts assembly.
<Exec Command='"$(SvcUtilPath)\svcutil" /dconly Schema\Contracts_v1_0.xsd /language:C# /serializable /directory:Types /namespace:http://contracts/v1.0,Contracts.Types.Contracts_v1_0' />
The contracts assembly is now done and we have a message which we can use in this sample.
Creating your message handler
In the server project in the sample we will hold our logic to handle the message and do whatever our application needs to with that message. To begin with we will create a class called GetCustomer_v1_0_Handler.
This class will implement the IHandleMessage interface so that it can handle messages and work in the AppFx.ServiceBus framework.
In addition to handling this interface we also need to add the MEF (managed extensibility framework) attributes so that during start up the host will identify this class as a message handler and workout what type of messages it handles.
The below snippet shows you what the class looks like when decorated with these attributes:
[Export(typeof(IHandleMessage))]
[ExportMetadata("MessageType", XsdMessageType)]
public class GetCustomer_v1_0_Handler : IHandleMessage
{
public const string XsdMessageType = "http://contracts/v1.0#GetCustomerRequest";
We now have a class which the framework will identify as handling the GetCustomerRequest message we defined earlier. I assume you have probably already read the earlier samples explaining what the different parts of the IHandleMessage interface relate, but if not please refer to the codeplex documentation page for more info.
In the message handler for our GetCustomer message in this sample we will only be handling two way messages so you will see the implementation in that class to look like below.
public object HandleRequestResponse(object message)
{
var request = message as Contracts.Types.Contracts_v1_0.GetCustomerRequest;
if (request == null)
{
Logger.Instance.Warn(string.Format("{0} recieved a null message", this.GetType().FullName));
throw new ApplicationException("Null message recieved")
{
RetrySupported = false
};
}
Logger.Instance.Info(string.Format("Message Id:{0}, Type={2} Handling Get Customer Request:{1}",
MessageProcessorContext.Current.MessageId, request.MessageID,
this.GetType().FullName));
throw new ApplicationException("I am throwing an error because this message handler is grumpy")
{
RetrySupported = false
};
}
For the purposes of the sample we will simply record that we received the message and then throw an error. To throw an error for RPC simply throw an AppFx.ServiceBus.Exceptions.ApplicationException. The exception allows you to specify if you wish to retry on the receive side. This is unlikely in an RPC scenario so you are most likely to specify false and return the framework will push an error response to the client.
Our message handler is now complete and ready to be used.
Configuring the receive side Host
The receive side configuration is exactly the same as for sample 2 which introduced the RPC pattern. Ill include the code snippets below but if you want to see more detail about the how and why of the configuration please refer to sample 2.
In the config file we have firstly declared the sections for AppFx.ServiceBus and also for log4net. As shown below:
Next we have the configuration for any connection strings we wish to use for connecting to the Windows Azure Service Bus namespace. In this case we only have one because most of the samples will run in the same namespace. The below picture shows you the connection string.
Next we will implement the AppFx.ServiceBus configuration. The main part to talk about in this sample is the listeners within the appfx.servicebus.receiver section pictured below.
<appfx.servicebus.receiver
defaultMaxRetries="3"
defaultMessageHandler="">
<listeners>
<add name="Default" connectionStringName="ServiceBusConnection" messagingEntity="Sample2-RPCRequest" noThreads="3"/>
</listeners>
<messageHandlers>
<!-- MEF used by default but this allows overriding of message handlers -->
</messageHandlers>
</appfx.servicebus.receiver>
Starting the receive side Host
To start the receive side host simple double click the AppFx.ServiceBus.Hosts.Console.exe application in the ..\Library\AppFx.ServiceBus.Hosts.Console folder.
Because you compiled the server project which copied over our contracts.dll containing our message types, the server.dll containing our message handlers and the AppFx.ServiceBus.Hosts.Console.exe.config file containing our configuration, you should see the console application will fire up and display some trace messages to show you that multiple listener instances are now polling the queue for messages.
The console application will look like the below:
The console is now waiting to process messages.
Configuring the client
In this sample you now need to refer to the client c# project in the solution. In this project you will see the app.config file which contains our configuration for AppFx.ServiceBus. The below picture shows this:
<configSections>
<section name="appfx.servicebus.client" type="AppFx.ServiceBus.Client.Configuration.ServiceBusConfiguration, AppFx.ServiceBus.Client"/>
<section name="log4net" type="log4net.Config.Log4NetConfigurationSectionHandler, log4net" />
</configSections>
<connectionStrings>
<add name="ServiceBusConnection" connectionString="Endpoint=sb://****.servicebus.windows.net/;SharedSecretIssuer=owner;SharedSecretValue=****
</connectionStrings>
<appfx.servicebus.client>
<clients>
<add name="Default" connectionString="ServiceBusConnection" sendToMessagingEntity="Sample2-RPCRequest" responseQueue="Sample2-RPCResponse" serializationFormat="Json"/>
</clients>
</appfx.servicebus.client>
The connection strings are the same as on the receive side where you will use the Windows Azure Service Bus connection string format.
In the appfx.servicebus.client element you can now add a number of clients. These clients can be referenced from your code to provide settings for sending a message. The example above specifies a client which will point to the Sample1-HelloWorld queue and send messages using JSON. In the default client you can see that the configuration is very similar to the one way message sample but the key difference is that we have supplied a responseQueue. This is the response queue which the framework will check for your RPC response.
Sending the message from the client
In the client application there is a windows form with a button which will allow you to test sending a message. The code behind this button is as follows:
private void button1_Click(object sender, EventArgs e)
{
var message = new Contracts.Types.Contracts_v1_0.GetCustomerRequest()
{
MessageID = Guid.NewGuid().ToString(),
CustomerId = "1"
};
var start = DateTime.Now;
var client =
new AppFx.ServiceBus.Client.SessionRequestResponseMessagingClient<Contracts.Types.Contracts_v1_0.GetCustomerRequest, Contracts.Types.Contracts_v1_0.GetCustomerResponse>("Default");
try
{
var response = client.SendRequestResponse(message);
var duration = DateTime.Now - start;
MessageBox.Show(string.Format("A successful response was recieved!, latency={0}ms", duration.TotalMilliseconds));
}
catch (AppFx.ServiceBus.Exceptions.ApplicationException appEx)
{
var duration = DateTime.Now - start;
MessageBox.Show(string.Format("Error recieved: {1}, latency={0}ms", duration.TotalMilliseconds, appEx.Message));
}
}
You can see that you will use the Generic MessagingClient class where you will tell it the two message types you wish to work with. You will also supply the client name which refers to your settings in the configuration file.
Next you use the SendRequestResponse method, and that will dispatch the message onto the queue.
When you click the button the message will be received by the server component. The component will throw an error as we coded earlier in the message handler. The error will come back across the response queue and the client side framework will throw an AppFx.ServiceBus.Exceptions.ApplicationException to the calling code. The above example shows how the client has caught this error and displayed it to the user.
When the message is flown over the queues or topic/subscriptions the framework will include an IsError property on the response message so the client framework waiting for the response message can work out the message is an error and the Service Bus message body will contain the error message.
In the sample the client will display a message like the following:
On the console application side you will spot that very soon after the message is submitted the console text will change to show that your message handler has executed. The console text will also show in red the error being logged on the server side and in yellow that the error has been returned to the client.
Summary
Hopefully this article shows you how much simpler AppFx.ServiceBus can make it for you to use Windows Azure Service Bus with the RPC pattern and including the handling or errors.
Sunday, February 3, 2013
In this article I will discuss implementing the RPC pattern using the AppFx.ServiceBus framework. In this scenario we want to achieve the following:
- Setup Windows Azure Service Bus to support this sample
- Define a schema for our message and generate the .net types which represent it
- Create a message handler to handle the message
- Start the host to listen for messages
- Create a client application to send the message
- Sit back and enjoy how easy it was J
Let's get on with things.
Setting up Azure Service Bus
In this sample we will use two queues. The first queue will be one which will be listened to by the server side host. This queue will be called Sample2-RPCRequest. You can create all of the appropriate queues for the sample by importing the Appfx-Servicebus-Samples_Entities.xml file which comes with the source code.
There is nothing particularly special about the Sample2-RPCRequest queue.
We will also create another queue called Sample2-RPCResponse. This queue is used when responses are sent from the server back to the client. One thing to note about this queue is that it has the "Requires Session" property set to true. This is so that the queue can receive the responses for multiple requests and have different listeners concurrently listening to the queue for their own response by using a session (note this is taken care of by the framework).
Creating your schema & message
When creating our message with AppFx.ServiceBus we prefer the technique of creating a schema to represent your message. This means we have a good contract for the message which we can then create classes in most programming languages to represent these schemas. This also allows us to serialize an object based on this schema to numerous formats such as XML or JSON.
One convention which we do use which is quite important is borrowed from BizTalk world where we use the combination of xml namespace + root element name to uniquely identify a message. The AppFx.ServiceBus framework will use this to be able to work out how to handle a message and how it can be serialized and deserialized.
In this example there are two messages (the request and the response), the message types are
http://contracts/v1.0#GetCustomerRequest
http://contracts/v1.0#GetCustomerResponse
Note: Please note the # character used to separate the namespace and root element name
In the sample we will begin in the Contracts c# project. Here we have created a schema in the schemas folder called contracts_v1_0.xsd and we have populated this with a simple definition of a type shown below:
<?xml
version="1.0"
encoding="utf-16"?>
<xs:schema
id ="Contracts_v1_0"
xmlns="http://contracts/v1.0"
elementFormDefault="qualified"
targetNamespace="http://contracts/v1.0"
xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element
name="GetCustomerRequest">
<xs:complexType>
<xs:sequence>
<xs:element
name="MessageID"
type="xs:string" />
<xs:element
name="CustomerId"
type="xs:string" />
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element
name="GetCustomerResponse">
<xs:complexType>
<xs:sequence>
<xs:element
name="CorrelationId"
type="xs:string" />
<xs:element
name="CustomerId"
type="xs:string" />
<xs:element
name="CustomerName"
type="xs:string" />
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:schema>
Now you would be able to give this schema to other parties who should be able to create classes from it and use them, but in our very simple demo we will next generate a class from this schema and keep it in the contracts assembly which will be shared either side of the queue just to keep it simple.
In the overridebuild.targets file you will see how we generate the class using svcutil which will produce a class in our contracts assembly.
<Exec
Command='"$(SvcUtilPath)\svcutil" /dconly Schema\Contracts_v1_0.xsd /language:C# /serializable /directory:Types /namespace:http://contracts/v1.0,Contracts.Types.Contracts_v1_0' />
The contracts assembly is now done and we have a message which we can use in this sample.
Creating your message handler
In the server project in the sample we will hold our logic to handle the message and do whatever our application needs to with that message. To begin with we will create a class called GetCustomer_v1_0_Handler.
This class will implement the IHandleMessage
interface so that it can handle messages and work in the AppFx.ServiceBus framework.
In addition to handling this interface we also need to add the MEF (managed extensibility framework) attributes so that during start up the host will identify this class as a message handler and workout what type of messages it handles.
The below snippet shows you what the class looks like when decorated with these attributes:
[Export(typeof(IHandleMessage))]
[ExportMetadata("MessageType", XsdMessageType)]
public
class
GetCustomer_v1_0_Handler : IHandleMessage
{
public
const
string XsdMessageType = "http://contracts/v1.0#GetCustomerRequest";
We now have a class which the framework will identify as handling the GetCustomerRequest message we defined earlier. I assume you have probably already read the earlier samples explaining what the different parts of the IHandleMessage interface relate, but if not please refer to the codeplex documentation page for more info.
In the message handler for our GetCustomer message in this sample we will only be handling two way messages so you will see the implementation in that class to look like below.
public
object HandleRequestResponse(object message)
{
var request = message as Contracts.Types.Contracts_v1_0.GetCustomerRequest;
if (request == null)
{
Logger.Instance.Warn(string.Format("{0} recieved a null message", this.GetType().FullName));
throw
new
ApplicationException("Null message recieved")
{
RetrySupported = false
};
}
Logger.Instance.Info(string.Format(
"Message Id:{0}, Type={2} Handling Get Customer Request: {1}",
MessageProcessorContext.Current.MessageId, request.MessageID,
this.GetType().FullName));
var response = new Contracts.Types.Contracts_v1_0.GetCustomerResponse()
{
CorrelationId = request.MessageID,
CustomerId = request.CustomerId,
CustomerName = "Mike"
};
return response;
}
For the purposes of the sample we will simply record that we received the message and return a response.
Our message handler is now complete and ready to be used.
Note: In the implementation above the logger from the framework will instrument into log4net which is used by the AppFx.ServiceBus framework.
Configuring the receive side Host
Next we will configure the AppFx.ServiceBus console host. We could just as easily use the Windows Service Host (which you would be more likely to do in a production scenario) however for the purposes of the demo we will use the console host.
In the server c# project you will see a file called AppFx.ServiceBus.Hosts.Console.exe. This is the file which needs configuration for the host to run and we have kept a copy of it with the server project to make it easier to see how this works.
In this file we currently have two things. There is the configuration for log4net which we will not go into except to say that log messages will be written to the console and also a rolling log file to help you with troubleshooting.
The rest of the configuration is for the receive framework for AppFx.ServiceBus.
In the config file we have firstly declared the sections for AppFx.ServiceBus and also for log4net. As shown below:
Next we have the configuration for any connection strings we wish to use for connecting to the Windows Azure Service Bus namespace. In this case we only have one because most of the samples will run in the same namespace. The below picture shows you the connection string.
Next we will implement the AppFx.ServiceBus configuration. The main part to talk about in this sample is the listeners within the appfx.servicebus.receiver section pictured below.
<appfx.servicebus.receiver
defaultMaxRetries="3"
defaultMessageHandler="">
<listeners>
<add
name="Default"
connectionStringName="ServiceBusConnection"
messagingEntity="Sample2-RPCRequest"
noThreads="3"/>
</listeners>
<messageHandlers>
<!-- MEF used by default but this allows overriding of message handlers -->
</messageHandlers>
</appfx.servicebus.receiver>
This listeners collection allows you to configure the host to listen to multiple service bus endpoints to process messages. In this example we will listen to the queue called Sample1-HelloWorld using the connection string we defined earlier and we will listen with 3 threads.
At this point if you compile the Server c# project I have modified the build output of this project so that the dll's and the config file will be copied to the ..\Library\AppFx.ServiceBus.Hosts.Console folder which is where the console application is located ready to run.
Starting the receive side Host
To start the receive side host simple double click the AppFx.ServiceBus.Hosts.Console.exe application in the ..\Library\AppFx.ServiceBus.Hosts.Console folder.
Because you compiled the server project which copied over our contracts.dll containing our message types, the server.dll containing our message handlers and the AppFx.ServiceBus.Hosts.Console.exe.config file containing our configuration, you should see the console application will fire up and display some trace messages to show you that multiple listener instances are now polling the queue for messages.
The console application will look like the below:
The console is now waiting to process messages.
Configuring the client
In this sample you now need to refer to the client c# project in the solution. In this project you will see the app.config file which contains our configuration for AppFx.ServiceBus. The below picture shows this:
<configSections>
<section
name="appfx.servicebus.client"
type="AppFx.ServiceBus.Client.Configuration.ServiceBusConfiguration, AppFx.ServiceBus.Client"/>
<section
name="log4net"
type="log4net.Config.Log4NetConfigurationSectionHandler, log4net" />
</configSections>
<connectionStrings>
<add
name="ServiceBusConnection"
connectionString="Endpoint=sb://****.servicebus.windows.net/;SharedSecretIssuer=owner;SharedSecretValue=****"/>
</connectionStrings>
<appfx.servicebus.client>
<clients>
<add
name="Default"
connectionString="ServiceBusConnection"
sendToMessagingEntity="Sample2-RPCRequest"
responseQueue="Sample2-RPCResponse"
serializationFormat="Json"/>
</clients>
</appfx.servicebus.client>
The connection strings are the same as on the receive side where you will use the Windows Azure Service Bus connection string format.
In the appfx.servicebus.client element you can now add a number of clients. These clients can be referenced from your code to provide settings for sending a message. The example above specifies a client which will point to the Sample1-HelloWorld queue and send messages using JSON. In the default client you can see that the configuration is very similar to the one way message sample but the key difference is that we have supplied a responseQueue. This is the response queue which the framework will check for your RPC response.
Sending the message from the client
In the client application there is a windows form with a button which will allow you to test sending a message. The code behind this button is as follows:
private
void button1_Click(object sender, EventArgs e)
{
var message = new Contracts.Types.Contracts_v1_0.GetCustomerRequest()
{
MessageID = Guid.NewGuid().ToString(),
CustomerId = "1"
};
var start = DateTime.Now;
var client = new AppFx.ServiceBus.Client.SessionRequestResponseMessagingClient<Contracts.Types.Contracts_v1_0.GetCustomerRequest, Contracts.Types.Contracts_v1_0.GetCustomerResponse>("Default");
var response = client.SendRequestResponse(message);
var duration = DateTime.Now - start;
MessageBox.Show(string.Format("Customer Name is: {1}, latency={0}ms", duration.TotalMilliseconds, response.CustomerName));
}
You can see that you will use the Generic MessagingClient class where you will tell it the two message types you wish to work with. You will also supply the client name which refers to your settings in the configuration file.
Next you use the SendRequestResponse method, and that will dispatch the message onto the queue.
When you click the button you will get a message box confirming that the message was sent and a response received.
On the console application side you will spot that very soon after the message is submitted the console text will change to show that your message handler has executed. The text indicates that an get customer message was received and processed and then a response returned.
Summary
Hopefully this article shows you how much simpler AppFx.ServiceBus can make it for you to use Windows Azure Service Bus. The framework should handle all of the plumbing and once your initial configuration of the Azure Service Bus is complete you will be able to focus on your functional code which will involve:
- Define message
- Create message handler
- Send the message and wait for response
In this article we will introduce a very simple messaging scenario which we will implement using the AppFx.ServiceBus framework to show how easy it can be to get up and running with Windows Azure Service Bus.
In this scenario we want to achieve the following:
- We want to define a schema for our message which will flow over the wire and generate .net types which can be used to represent this object
- From a .net client application we want to push a message to a queue in Windows Azure Service Bus
- In this example we will use JSON serialization format
- We then want to use the AppFx.ServiceBus console host application to be configured to handle our message
- We will develop a message handler which will be dll dropped into the console hosts bin directory which will contain our custom code
Hopefully that is pretty clear, so now lets walk through the steps to implement it.
Setting up Azure Service Bus
The Windows Azure configuration for this particular sample is very easy. We simple need a queue and to ensure we have permissions to access it to send and receive messages.
I'm going to assume at this stage you are already familiar with how to create queues in Windows Azure Service Bus and you know about being able to use the owner credential which is ok for demo purposes.
For this sample we will create a new queue called Sample1-HelloWorld.
The rest of the settings on this queue can remain the default settings.
Creating your message
When creating our message with AppFx.ServiceBus we prefer the technique of creating a schema to represent your message. This means we have a good contract for the message which we can then create classes in most programming languages to represent these schemas. This also allows us to serialize an object based on this schema to numerous formats such as XML or JSON.
One convention which we do use which is quite important is borrowed from BizTalk world where we use the combination of xml namespace + root element name to uniquely identify a message. The AppFx.ServiceBus framework will use this to be able to work out how to handle a message and how it can be serialized and deserialized.
An example of this message type from this sample is:
http://contracts/v1.0#UpdateCustomer
Note: Please note the # character used to separate the namespace and root element name
In the sample we will begin in the Contracts c# project. Here we have created a schema in the schemas folder called contracts_v1_0.xsd and we have populated this with a simple definition of a type shown below:
<?xml
version="1.0"
encoding="utf-16"?>
<xs:schema
id ="Contracts_v1_0"
xmlns="http://contracts/v1.0"
elementFormDefault="qualified"
targetNamespace="http://contracts/v1.0"
xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element
name="UpdateCustomer">
<xs:complexType>
<xs:sequence>
<xs:element
name="MessageID"
type="xs:string" />
<xs:element
name="Name"
type="xs:string" />
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:schema>
Now you would be able to give this schema to other parties who should be able to create classes from it and use them, but in our very simple demo we will next generate a class from this schema and keep it in the contracts assembly which will be shared either side of the queue just to keep it simple.
In the overridebuild.targets file you will see how we generate the class using svcutil which will produce a class in our contracts assembly.
<Exec
Command='"$(SvcUtilPath)\svcutil" /dconly Schema\Contracts_v1_0.xsd /language:C# /serializable /directory:Types /namespace:http://contracts/v1.0,Contracts.Types.Contracts_v1_0' />
The contracts assembly is now done and we have a message which we can use in this sample.
Note: If you look in the other samples we will be likely to use this same technique elsewhere so it's worth taking a minute to understand this.
Creating your message handler
In the server project in the sample we will hold our logic to handle the message and do whatever our application needs to with that message. To begin with we will create a class called UpdateCustomer_v1_0_Handler.
This class will implement the IHandleMessage
interface so that it can handle messages and work in the AppFx.ServiceBus framework.
In addition to handling this interface we also need to add the MEF (managed extensibility framework) attributes so that during start up the host will identify this class as a message handler and workout what type of messages it handles.
The below snippet shows you what the class looks like when decorated with these attributes:
[Export(typeof(IHandleMessage))]
[ExportMetadata("MessageType", XsdMessageType)]
public
class
UpdateCustomer_v1_0_Handler : IHandleMessage
{
public
const
string XsdMessageType = "http://contracts/v1.0#UpdateCustomer";
We now have a class which the framework will identify as handling the UpdateCustomer message we defined earlier.
The below picture shows the definition of our message handler class.
Just to explain some of those elements of the interface in a little more detail.
Interface Item | Description |
HandleRequestResponse | This is the method where your implementation will go if your message handler gets a request response message |
Handle
| This is the method where your implementation will go if your message handler gets a one way message
|
InputMessageType
| This is the string defining the message type your message handler handles. This is in the xml namespace + root element format
|
InputClrMessageType
| This is the .net message type which your message handler wants the message deserialized to. This will be the type of the class you generated from the schema
|
HandlesRequestResponseMessages
| This is a Boolean indicator to inform the framework if the message handler handles request response messages
|
HandlesOneWayMessages
| This is a Boolean indicator to inform the framework if the message handler handles one way messages
|
Priority
| This is the priority used to workout which message handler to execute if two are in the bin directory handling the same message. The highest priority is executed and others are not.
|
DeserializeRequestMessage
| This indicates if the message handler wants the AppFx.ServiceBus framework to deserialize the request message before handing over execution to the handler. Normally this may be true unless you are executing a handler which is generic
|
In the message handler for our customer update message in this sample we will only be handling one way messages so you will see the implementation in that class to look like below.
For the purposes of the sample we will simply record that we received the message.
Our message handler is now complete and ready to be used.
Note: In the implementation above the logger from the framework will instrument into log4net which is used by the AppFx.ServiceBus framework.
Configuring the receive side Host
Next we will configure the AppFx.ServiceBus console host. We could just as easily use the Windows Service Host (which you would be more likely to do in a production scenario) however for the purposes of the demo we will use the console host.
In the server c# project you will see a file called AppFx.ServiceBus.Hosts.Console.exe. This is the file which needs configuration for the host to run and we have kept a copy of it with the server project to make it easier to see how this works.
In this file we currently have two things. There is the configuration for log4net which we will not go into except to say that log messages will be written to the console and also a rolling log file to help you with troubleshooting.
The rest of the configuration is for the receive framework for AppFx.ServiceBus.
In the config file we have firstly declared the sections for AppFx.ServiceBus and also for log4net. As shown below:
Next we have the configuration for any connection strings we wish to use for connecting to the Windows Azure Service Bus namespace. In this case we only have one because most of the samples will run in the same namespace. The below picture shows you the connection string.
Next we will implement the AppFx.ServiceBus configuration. The main part to talk about in this sample is the listeners within the appfx.servicebus.receiver section pictured below.
This listeners collection allows you to configure the host to listen to multiple service bus endpoints to process messages. In this example we will listen to the queue called Sample1-HelloWorld using the connection string we defined earlier and we will listen with 3 threads.
The rest of the elements in this configuration we will go through in future samples.
At this point if you compile the Server c# project I have modified the build output of this project so that the dll's and the config file will be copied to the ..\Library\AppFx.ServiceBus.Hosts.Console folder which is where the console application is located ready to run.
Starting the receive side Host
To start the receive side host simple double click the AppFx.ServiceBus.Hosts.Console.exe application in the ..\Library\AppFx.ServiceBus.Hosts.Console folder.
Because you compiled the server project which copied over our contracts.dll containing our message types, the server.dll containing our message handlers and the AppFx.ServiceBus.Hosts.Console.exe.config file containing our configuration, you should see the console application will fire up and display some trace messages to show you that multiple listener instances are now polling the queue for messages.
The console application will look like the below:
The console is now waiting to process messages.
Configuring the client
In this sample you now need to refer to the client c# project in the solution. In this project you will see the app.config file which contains our configuration for AppFx.ServiceBus. The below picture shows this:
The connection strings are the same as on the receive side where you will use the Windows Azure Service Bus connection string format.
In the appfx.servicebus.client element you can now add a number of clients. These clients can be referenced from your code to provide settings for sending a message. The example above specifies a client which will point to the Sample1-HelloWorld queue and send messages using JSON.
Sending the message from the client
In the client application there is a windows form with a button which will allow you to test sending a message. The code behind this button is as follows:
You can see that you will use the Generic MessagingClient class where you will tell it what type of message you wish to send. You will also supply the client name which refers to your settings in the configuration file.
Next you use the send method, and that will dispatch the message onto the queue.
When you click the button you will get a message box confirming the message has been sent which looks like the below:
On the console application side you will spot that very soon after the message is submitted the console text will change to show that your message handler has executed. The text indicates that an update customer message was received and processed and then the host started listening for the next message.
Summary
Hopefully this article shows you how much simpler AppFx.ServiceBus can make it for you to use Windows Azure Service Bus. The framework should handle all of the plumbing and once your initial configuration of the Azure Service Bus is complete you will be able to focus on your functional code which will involve:
- Define message
- Create message handler
- Send the message
Wednesday, January 30, 2013
AppFx.ServiceBus is a new framework we have developed based on some real world experience on projects involving Windows Azure Service Bus queues and topics. The overall aims of the project are as follows:
In our experience we found that when working on projects we had one of 2 choices when it came to processing messages from the Windows Azure Service Bus. Firstly there was the option of using WCF but we found that while this was relatively simple to get up and running it was difficult to implement complex scenarios and have control over lower level things. In essence we felt that WCF processing from Azure Service Bus didn't fit many of our scenarios. The second option was to write your own windows service which polled messages and then did stuff with them. The down side of this was that most of the samples on how to do this simply dealt with one type of message and this lower level plumbing code could be time consuming to build and get right.
Using AppFx.ServiceBus you can use one of the hosts provided with the framework to do the listening for you. You simply need to supply some simple configuration settings and then start the host and it will do the hard work for you.
This then leaves the developer to focus on writing message handlers which can be used to implement the custom logic for each type of message.
AppFx.ServiceBus comes with a Windows Service Host and a Console Application host. The console application is really intended to make it easier during the development stages of your project whereas the Windows Service is intended for production scenarios. You can also install multiple instances of the windows service and implement different separation scenarios which will be described through the samples which will accompany AppFx.ServiceBus.
For more info please look at the Codeplex site:
http://appfxservicebus.codeplex.com/
Wednesday, January 23, 2013
Recently at the BizTalk Summit in London I was asked a question about Message Archiving from BizTalk and one of the most common solutions to this is the Message Archiving Pipeline Component which was written by my old friend Nick Heppleson.
After the summit I was pondering this archiving feature and wish that at the time I had mentioned Storsimple. Recently Id been speaking with Michael Royster from Microsoft in the UK and he had been telling me about the new acqusition Microsoft had made and how this solution combines on-premise storage with storage in Windows Azure which offers lots of opportunities.
The key thing here is that Storsimple is an appliance which you add to your data centre which offers up file storage but only certain data is kept on premise and the rest is kept in the cloud. The appliance handles the magic underneath that but your applications just see file shares on the network which they can communicate with.
Coming back to the archiving requirement if you have a customer who needs to archive a lot of messages and your worried about house keeping around this then you should certainly consider combining Message Archive Pipeline Component plus StorSimple to provide and excellent combined solution to this problem.
There is obviously a cost for StorSimple but you can reuse the applicance across other applications and use if for storage to help sharepoint and exchange implementations too or possibly back up archiving.
Anyway just a few thoughts that were lingering around my head on the train ride home last week. Heres some links if your interested:
http://www.storsimple.com/
http://www.microsoft.com/casestudies/Case_Study_Detail.aspx?CaseStudyID=4000008345
Tuesday, January 8, 2013
I thought I’d brain dump a few thoughts after the recent BizTalk summit and how some of the information will help shape solutions with the customers whom I work with.
BizTalk On-Premise
Traditionally for a Microsoft focused integration team BizTalk was often used as the platform for nearly all integration solutions and this was often taken to the degree where some companies would make enterprise architecture decisions that “all integration would be done in BizTalk”. As a product BizTalk often would allow you to do this because it has an extensive feature set which covers most of the core capabilities needed in an integration platform.
In recent years the Microsoft Integration Developer’s tool set has been extended with other products which have become more mature and now mean that BizTalk is not the only way to solve an integration with Microsoft technology and now the BizTalk product itself is expanding to include the IAAS and PAAS offerings which should change the solutions that we build in the future.
I believe that this will change the way that organisations use BizTalk as part of their integration platforms. I believe we will see fewer of those solve everything with BizTalk implementations and I believe we will see customers who have smaller BizTalk groups than we did in the past as they begin the embrace the use of other products along-side BizTalk.
I think BizTalk will also be used more to focus on the dirtier side of integration in the EAI space where you can use the many features of BizTalk and its adapter set to integrate into various line of business systems.
BizTalk IAAS
I think the benefits of the IAAS offering of BizTalk proposed in the next version of BizTalk will be particularly attractive to many organisations. This will probably fall into two categories. Firstly there will be those companies who are comfortable using the Windows Azure Service Bus and can bridge an on-premise and cloud instance of BizTalk using either the Service Bus Relay or Queue’s and Topics. This will allow them to use BizTalk in the cloud for things like burst or batch patterns and then use Service Bus to integrate messages into on premise applications. I can think of a number of customer scenarios where this option to process batch files in the cloud and to produce a queue of messages to process would be very attractive.
The second scenario would be for customers who embrace the networking capabilities of Windows Azure and connect their data centre to a Windows Azure data centre hosting their new BizTalk IAAS instance. This would allow for a greater use of the BizTalk feature set in the cloud and would mean you can integrate with applications exactly as you do now. This would be very attractive for many customers who might wish to minimize their on-premise infrastructure investment. One of the big benefits of the extended tool set which is available is that there are other alternatives for some of the traditional “as low as possible” latency solutions which opens up the option of moving some of all of your BizTalk investment off premise.
I would be surprised if in the future Microsoft didn’t offer pre-built templates for many of the different topologies BizTalk is often deployed in. This would be a massive win because it is often time consuming and expensive to get your BizTalk installation setup correctly and to prove it performs well. If this was a template, then all customers would be able to start with an environment that adheres to best practices rather than this being uncommon.
Customers using BizTalk IAAS will also have the ability to scale up and down their group on demand. When I think back to some customer scenarios where you need to scale for peak demand scenarios and then have a BizTalk environment which has plenty of capacity for large periods. These customers will eventually be able to automate scaling based on actual demand and predicted demand. The IAAS offering will also be likely to make the licensing scenarios simpler so that a customer can have multiple BizTalk groups whereas with traditional on-premise setups it might not be practical. I can think of one customer scenario where they have a weekend batch which requires the BizTalk group to require 4 BizTalk servers which are maxed out for the processing window of the batch, and then at other times this BizTalk group processes relatively low volume messaging. In this customers scenario it might be feasible to separate the BizTalk group into two groups. One small group of 2 BizTalk servers to provide highly available messaging but the batch stuff is on the 2nd group. This group could then only be on for the duration of the batch window and turned off after. Perhaps the 2nd group would utilize 6 BizTalk servers to process the batch and then be turned off the rest of the week.
Depending upon how Microsoft licenses the BizTalk IAAS offering, multiple group scenarios and part-time usage scenarios could be much more cost effective for customers that in the past.
I think it will also be interesting to see how BizTalk as an IAAS offering compares in its uptake to the way customers used to use BizTalk branch edition. In my experience I haven’t really come across any customers who have used branch edition and I always felt that its limitations meant that bigger customers would just get standard edition setups in their local offices. I would expect that companies who embrace the cloud may find BizTalk IAAS as an interesting alternative way of doing this.
BizTalk PAAS
The BizTalk PAAS offering also means that some of the traditional work done in an on-premise instance of BizTalk may be able to be done in the PAAS platform. The features available in the PAAS offering for BizTalk will initially be small by comparison with on-premise BizTalk so you will only be able to do a limited number of things, but over time this is bound to extend. I think its interesting that one of the key areas of focus is EDI and I believe that in recent years one of the key areas of growth for BizTalk was around EDI scenarios when BizTalk introduced a new version with some new EDI enhancements which other vendors had not already implemented. This is an example of where the time to market offered by the BizTalk PAAS platform could be a key element in the future. I would expect that other accelerators and industry vertical solutions would be areas where there is good value to be had in the PAAS platform, particularly if on-premise BizTalk plays well with the PAAS platform.
I think BizTalk PAAS will open up new opportunities which we have never had in the BizTalk space before. I think small organisations who would never have considered BizTalk or any of the other big integration vendors will now have the ability to develop structured integration solutions using the Microsoft platform. This will have the benefits that they will develop solutions which will have a better chance of being able to grow and scale as the organization grows. It will also help in acquisition scenarios when smaller organisations have built solutions on a platform rather than the spaghetti custom solutions im sure we have all come across before.
I think BizTalk PAAS will also help for B2B scenarios with regard to organisations passing data between each other in a secure way.
BizTalk for Large Organisations
I think that in larger organisations you will tend to see smaller BizTalk setups than usual but also potentially more organisations with more than one BizTalk group. These will still be some large organisations who do those massive BizTalk implementations with specialist requirements which occasionally come up but they will probably do more hybrid solutions using BizTalk plus other Microsoft integration technologies.
There are likely to be many organisations who will not really change much for many years, but at the same time there will be others who embrace some of the new opportunities.
I think in that the organisations who really take advantage of these changes you will find that they will have moved some of their existing BizTalk investment to the BizTalk IAAS offering because it offers cost savings. I would expect they will still keep some BizTalk on premise but its likely to be smaller scale and to have a few constraints around what kind of patterns are implemented on premise to ensure they need to be on premise.
I think the most effective larger organisations will be likely to use most of the newer Microsoft integration technologies for example:
- On-premise
o BizTalk
o Windows Server Service Bus
o Workflow Host
- Cloud
o BizTalk IAAS
o BizTalk PAAS
o Windows Azure Service Bus
These organisations will have a great platform which will allow them to use the best technology suited to each integration problem and hopefully the integration between the technologies in the platform will be very seamless.
BizTalk for Medium Sized Organisations
Medium sized organisations are bound to be interesting places to work. Typically you come across those who try to implement BizTalk but struggle for various reasons. They usually struggle around the ability to get skilled resources and to implement and manage a good infrastructure. I would expect that in the future BizTalk IAAS will be one of the attractive bits to these companies because it means some of the problem areas they typically face can be removed. I would also expect that the IAAS offering will help many organisations find BizTalk is more affordable if it moves towards a usage based cost model. These organisations may not have been able to afford a suitable BizTalk setup and implementation previously but when your deployment and management costs get smaller and you have the ability to pay based on CPU hour rather than the full cost up front then this becomes much more feasible for many companies.
There will still be medium sized organisations who do BizTalk just as they do now, but there will be many who can save money by considering the above and there will be those who have never had BizTalk before who suddenly become potential BizTalk users.
For some medium sized organisations they may find that they have BizTalk but don’t really need it. They would be the kind of organization where they implemented a BizTalk solution because it was previously Microsoft’s only real integration offering and it did most things pretty well. These organisations may find that they don’t necessarily need BizTalk anymore because there are other Microsoft technologies which do certain integration patterns slightly better or as well as BizTalk. For these customers they may find that they can get migrate their BizTalk investment to take advantage of other Microsoft technologies which meet their requirements better than BizTalk does but offer a better value for money solution. These organisations are typically ones where you might look at their BizTalk investment and feel it was overkill of the solution they were trying to achieve.
Medium sized companies are also more likely to consider BizTalk IAAS solutions because the risk profile around BizTalk projects will change significantly if the licensing becomes usage based. In the past a medium sized company would be taking a reasonable risk to purchase licenses for SQL and BizTalk during their project life cycle as an up-front cost. If the project failed to deliver its business value for some reason then they would still have purchased these licenses which would not be cheap. In a usage based scenario if the business project failed to deliver results then you can just turn off and remove the BizTalk virtual machines and stop paying for them. This would remove some of the barriers that would have previously been a blocker to companies considering a BizTalk project.
BizTalk for Small Organisations or Start-ups
Smaller organisations are one of the areas which I find most exciting. Often these companies will not consider BizTalk solutions because the cost to implement a solution makes it unfeasible. These companies are also some of the ones which can offer some of the most innovative opportunities. With the changes coming in the BizTalk space and the extended Microsoft integration platform including the other technologies discussed earlier, these types of company will be in a better position than ever to start using a structured integration platform with Microsoft from day one. This will create opportunities for Microsoft integration developers with companies who they have never worked with before.
Imagine small companies using an on-premise instance of Service Bus for Windows Server or BizTalk IAAS. They will be able to create solutions when the business is a company of less than 10 people which will be on infrastructure which could scale with the company when it grows into a company with thousands of employees. This scalability will allow them to take more iterative approaches to enhancing their integration platform rather than the typical distinct phases where a growing organisation makes periodic massive investments in IT projects because the organisation has outgrown their applications and integration solutions.
Conclusion
As I mentioned at the start of the article this has been a little bit of a brain dump of my thoughts following the BizTalk summit and Im hoping that I have articulated that I believe there some quite exciting opportunities ahead in the integration space around Microsoft technology and particularly with BizTalk.
I think one thing that did come to mind while I was at the BizTalk summit was the marketing story Microsoft used a couple of years ago with BizTalk which was “Better Together”. I believe this marketing story is still true and stronger than ever. I believe as architects using these technologies it is important to understand the capabilities of each and the overlaps and distinctions between them to help customers in the future leverage the right tool for the right job. There is a lot of talk around hybrid solutions but I think that in time people will appreciate that hybrid solutions do not just mean integrating on-premise with the cloud but also a hybrid integration solution can mean combining integration technologies that work together to deliver a solution regardless of where the technology execution is hosted.
I’m really looking to see what other people think and provoke some discussion around this
Tuesday, November 20, 2012
Incase who ever it is who reads my blog happens to want to know a little more about me, Steef-Jan prepared an interview with me in relation to technet wiki last week.
Its on the following link:
http://blogs.technet.com/b/wikininjas/archive/2012/11/19/interview-with-a-microsoft-integration-mvp-michael-stephenson.aspx
Saturday, November 10, 2012
I was having some very frustrating behaviour on our build server and while I found a number of articles online with similar error messages none of them helped me. I thought I would just explain this here incase if helps me or anyone else in future.
The error message we were getting is:
There was no endpoint listening at http://localhostStubs.ExternalApplication/SampleService.svc that could accept the message. This is often caused by an incorrect address or SOAP action. See InnerException, if present, for more details
Our scenario is as follows:
We have a solution where a WCF service application hosting the WCF routing service is listening to the Windows Azure Service Bus Relay. We have an acceptance test project in the solution which sends a message to the service bus which is then received by the WCF routing service and routed to SampleService.svc which is hosted in another IIS application on the same box. A response is flowed back through to the test.
In the tests there are 5 scenarios simulating a successful message, and various error conditions. On my developer machine it was working absolutely fine every time, and a clean build on my developer machine worked fine. On the build server however one or more of the tests would fail each time with the above error message. There didnt seem to be any pattern to which test would fail.
The solution was building on a Windows 2008 R2 machine with IIS 7 and AppFabric Server installed with auto-start configured for the IIS Application which would be listening to service bus.
After lots of searching online and looking at logs etc it turned out to be a simple solution to just restart the WAS service (Windows Process Activation Service) and the services it advised you to restart with it.
Hope this helps someone else