Build 2011 announced the September release of the Windows Azure Service Bus. For those of you who have just tuned in, the WA Service Bus is part of the AppFabric middleware that enables almost seamless connectivity between applications that are restricted to limited connectivity options by their firewall or custom security protocols. By leveraging the service bus one can build distributed applications on cloud or hybrid solutions featuring on-premise and cloud apps while maintaining the worthwhile security. But this isn’t an ‘Introduction to Service Bus’ blog post and I like to stick to the title of the blogs as often as I can without trying to distract myself with other stuff that is going on around me – like the dinosaur that just walked past my window – see, there I go again. Sorry. Back to the Service Bus. So, I was just exploring the shiny-new platform, and suddenly realized that there was no possible way I could NOT blog about this. Hallowed be the texts of GeeksAndTheirInternet. Here is an attempted brief summary of the new features released in Sept 2011 and the ways to go about playing with them.

Before we delve into that, wouldn’t it be more polite to outline the basic difference between the old WASB, the May 2011 CTP WASB and the Sept 2011 WASB releases? Yes it would. Lets see if I can suppress this in a single statement! The old WASB advocated only Relay Messaging capabilities while the May 2011 one boasts of relay as well as brokered messaging powers whereas the Sept 2011 release merely tweaked a couple of API’s here and there to fool the world into thinking that the AppFabric team had been productive for the past 6 months (more later). Phew! Just kidding Microsoft.    Uhhhh. What was that about the relay and brokered messaging? God! This is like the inception of differences. In Relay Messaging, a bidirectional socket is established between the on-premise service and the Service bus, post which the client can communicate with the Service bus to pass on the message. The catch here is that for effective communication both the producer and consumer need to be online throughout the communicative period. So, MS introduced brokered messaging in its May CTP, thereby providing messaging capabilities for decoupled scenarios like either the receiver or sender being offline at the same time. Here is a more elaborate differentiation.

Now that that’s behind us, lets move on to the code battlefield which is a bit more serene as compared to what that dinosaur has done to my backyard. The first thing that we need to do to avail the Azure Service Bus functionality is to create a Service Bus Namespace in the Windows Azure Portal. An Issuer Name and Issuer Key must be provided to do the same. Once that's done whip out your VS 2010.

For starters lets take a look at Relayed Messaging. Nothing much has changed here. The steps to be followed to achieve a connection between the producer and consumer are as follows:

For the sender:

1. Create the Service contract and its implementation.

2. Create the service URI based on the service namespace
Uri address = ServiceBusEnvironment.CreateServiceUri("sb", <service namespace that you provided in the portal>, <your service name>);

3. Create the credentials object for the endpoint
TransportClientEndpointBehavior sharedSecretServiceBusCredential = new TransportClientEndpointBehavior();
sharedSecretServiceBusCredential.TokenProvider = TokenProvider.CreateSharedSecretTokenProvider(<issuer name>, <issuer key>);

This is one of the changes that have been implemented in the September release. The credentials for the service bus are provided using a special TokenProvider class instead of being provided directly to the API. This is closer to the actual connection implementation mechanism of the Service Bus wherein the access tokens required to access the SB are provided by the Access Control Service (ACS) of the AppFabric.           
           
4. Create the service host at that address
ServiceHost host = new ServiceHost(typeof(<service name>), address);

5. Create the ServiceRegistrySettings behavior for the endpoint
IEndpointBehavior serviceRegistrySettings = new ServiceRegistrySettings(DiscoveryType.Public);

The Service Bus maintains a registry for all the endpoints that connect to it. Any guesses as to what its called? Oh, Right. The ServiceRegistrySettings can specify whether the endpoint should be private or public. If the DiscoveryType is public the endpoint saved to the ServiceRegistry would be published to its ATOM feed.  

6. Add the Service Bus credentials to all endpoints specified in configuration
foreach (ServiceEndpoint endpoint in host.Description.Endpoints)
{
endpoint.Behaviors.Add(serviceRegistrySettings);
endpoint.Behaviors.Add(sharedSecretServiceBusCredential);
}
           
7. Open the service
host.Open();

For Receiver:

1. Establish a channel interface for your services                                                                   

public IYourChannel : <Your Service Contract>, IClientChannel                                                         

{}

2. Create the service URI based on the service namespace
Uri address = ServiceBusEnvironment.CreateServiceUri("sb", <service namespace that you provided in the portal>, <your service name>);           

3. Create the credentials object for the endpoint
TransportClientEndpointBehavior sharedSecretServiceBusCredential = new TransportClientEndpointBehavior();
sharedSecretServiceBusCredential.TokenProvider = TokenProvider.CreateSharedSecretTokenProvider(<issuer name>, <issuer key>);

4. Create the channel factory loading the configuration
ChannelFactory<IYourChannel> channelFactory = new ChannelFactory<IYourChannel>("RelayEndpoint", new EndpointAddress(serviceUri));                                                                  

5. Apply the Service Bus credentials 
channelFactory.Endpoint.Behaviors.Add(sharedSecretServiceBusCredential);

6. Create and open the client channel 
IYourChannel channel = channelFactory.CreateChannel(); 
channel.Open();

7. Once the channel is opened, the service is ready to be consumed.

Now lets look at the Brokered Messaging – the relatively new guy on the block. As mentioned earlier Brokered Messaging relies on the fact that both the producer and consumer need NOT be online at the same time for the successful transfer of messages. Advantages? Asynchronous messaging capabilities since the sender can issue messages without waiting for a response from the receiver using Queues model. Improved Load Balancing as multiple subscribers can be involved for a single Topic using the Topic-Subscription model (expect elaboration). 

Let us see how we can implement the Brokered Messaging using the Queue Model:

For Sender:

1. Create Tokens for the sender using TokenProvider class.
TokenProvider tkp = TokenProvider.CreateSharedSecretKeyTokenProvider(IssuerName, IssuerKey);
2. Create Service Bus URI
Uri serviceUri = ServiceBusEnvironment.CreateServiceUri("sb", Sender.ServiceNamespace, string.Empty);
3. Initialize NamespaceManager
NamespaceManager namespaceClient = new NamespaceManager(serviceUri, tkp); [NamespaceManager is another change that has been introduced in the September release.]
4. Using this NamespaceManager create a Queue
namespaceClient.CreateQueue("<any queue name>");
5. Create a MessagingFactory
MessagingFactory factory = MessagingFactory.Create(serviceUri, credentials);
6. Using this factory create a QueueClient
QueueClient myQueueClient = factory.CreateQueueClient("<same queue name as specified above>");
7. Create Brokered Message(s) which is to be sent
BrokeredMessage message = new BrokeredMessage("<message body>");
message.MessageId = "<message ID>";
8. Send the brokered message using QueueClient created above
myQueueClient.Send(message);
[If there are more than one brokered messages create a list of them and while iterating through the list send each of them via the Queue]

For Receiver:

1. Create Tokens for the sender using TokenProvider class.
TokenProvider tkp = TokenProvider.CreateSharedSecretKeyTokenProvider(IssuerName, IssuerKey);
2. Create Service Bus URI
Uri serviceUri = ServiceBusEnvironment.CreateServiceUri("sb", Sender.ServiceNamespace, string.Empty);
3. Initialize NamespaceManager
NamespaceManager namespaceClient = new NamespaceManager(serviceUri, tkp);
4. Using this NamespaceManager retrieve the queue that was created by the sender
QueueDescription queueDescription = namespaceClient.GetQueue("<same queue name as given by sender>");
5. Create a MessagingFactory
MessagingFactory factory = MessagingFactory.Create(serviceUri, credentials);
6. Using this factory create a QueueClient
QueueClient myQueueClient = factory.CreateQueueClient("<same queue name>", ReceiveMode.PeekLock);
[ReciveMode specifies the way in which the message should be handled]
7. Create Brokered Message(s) which is to be recieved and receive the same
BrokeredMessage message = myQueueClient.Receive(TimeSpan.FromSeconds(5));
[If there are more than one messages to be recieved, iterate for the queueDescription.MessageCount and then receive each message]

Now, to look into the Topic-Subscription model of communication

For Sender:

1. Create Tokens for the sender using TokenProvider class.
TokenProvider tkp = TokenProvider.CreateSharedSecretKeyTokenProvider(IssuerName, IssuerKey);
2. Create Service Bus URI
Uri serviceUri = ServiceBusEnvironment.CreateServiceUri("sb", Sender.ServiceNamespace, string.Empty);
3. Initialize NamespaceManager
NamespaceManager namespaceClient = new NamespaceManager(serviceUri, tkp);
4. Using this NamespaceManager create a Topic
TopicDescription myTopic = = namespaceClient.CreateTopic("<any topic name>");
5. Create Subscriptions using NamespaceManager
SubscriptionDescription myAuditSubscription = namespaceManager.CreateSubscription(myTopic.Path, "AuditSubscription");
SubscriptionDescription myAgentSubscription = namespaceManager.CreateSubscription(myTopic.Path, "AgentSubscription");
6. Create a MessagingFactory
MessagingFactory factory = MessagingFactory.Create(serviceUri, credentials);
7. Using this factory create a TopicClient
TopicClient myTopicClient = factory.CreateTopicClient(myTopic.Path);
8. Create Brokered Message(s) which is to be sent
BrokeredMessage message = new BrokeredMessage("<message body>");
message.MessageId = "<message ID>";
9. Send the brokered message using TopicClient created above
myTopicClient.Send(message);
[If there are more than one brokered messages create a list of them and while iterating through the list send each of them via the Topic]

For Receiver:

1. Create Tokens for the sender using TokenProvider class.
TokenProvider tkp = TokenProvider.CreateSharedSecretKeyTokenProvider(IssuerName, IssuerKey);
2. Create Service Bus URI
Uri serviceUri = ServiceBusEnvironment.CreateServiceUri("sb", Sender.ServiceNamespace, string.Empty);
3. Initialize NamespaceManager
NamespaceManager namespaceClient = new NamespaceManager(serviceUri, tkp);
4. Create a MessagingFactory
MessagingFactory factory = MessagingFactory.Create(serviceUri, credentials);
5. Create SubscriptionClients for each subscription created by the sender
SubscriptionClient agentSubscriptionClient = factory.CreateSubscriptionClient("<Topic name as specified by sender>", "AgentSubscription", ReceiveMode.PeekLock);
SubscriptionClient auditSubscriptionClient = factory.CreateSubscriptionClient("<Topic name as specified by sender>", "AuditSubscription", ReceiveMode.ReceiveAndDelete);
5. Create Brokered Message(s) which is to be recieved and receive the same
BrokeredMessage messageAgent = agentSubscriptionClient.Receive(TimeSpan.FromSeconds(5))
BrokeredMessage messageAudit = auditSubscriptionClient.Receive(TimeSpan.FromSeconds(5))
[For more than one messages you can iterate a while loop until the message recieved is null]

So, we can create as many subscriptions for a single topic. And these subscriptions can be availed by the clients at any point of time.

A more comprehensive and real life example would be the Retail web service provided by Akite which uses this new topic-sub broker model of to sync POS clients with the headquarters -http://www.akite.net/en/PosNet.aspx. This example is also outlined in the Clemens Vaster’s Build 2011 video.

Well, this just about covers the WASB Sept 2011 release. You can download the samples from the codeplex site. Alright, now if you will excuse me, I have to kick some dino butt!! Ma, where my shotgun at?

I have had a total of 2 years, 3 months, 2 hours of Windows Azure developer experience and it humiliates me to suicidal extents to confess that I am hardly conversant with the platform’s native Diagnostics technicalities. “I am Tinu Thomas and I am a diagnostic-ignorant Azure Developer.” Oh my god…how could he?? Now that we have that past us <sheepish grin>, I would also like to bask in the afterglow of redemption. Thanks to a much-needed job transition phase, I was able to find enough time to explore my forbidden fantasies regarding Diagnostics and eventually came to a foregone conclusion – I “need” to blog about this, irrespective of whether or not it’s a tried and tested topic, it was my moral responsibility as a mortal developer to reiterate my freshly acquired knowledge to my other fellow kind, despite the fact that they may already be well aware and much more well-informed than me on the subject, because it is written in the holy texts of GeeksAndTheirInternet – ‘Thou shall blog what thou hath painstakingly learned’. Amen.

So how does WAD work?

So, before your Role ‘starts’, a process called as a Diagnostics Monitor latches itself on to  the role (think Aliens and you got it). Lets say that this monitor has already been ‘configured’ (more on this later). Once the role starts, it goes about its usual activity <yada-yada-yada> all the while being stealthily screened by our DM for any errors, information or logs which are in turn stored within the role or VM’s local storage.These logs are then meticulously transferred to Azure Storage at intervals. So that’s basically the gist of it. Not that bad huh?

Well, put on your gas masks… We are going in!

When I mentioned that the monitor was ‘configured’, what did I mean? Well like every other process, Diagnostics Monitor also needs to be told what to do. Configuring the DM is like instructing it to follow a set of rules – for example: what exactly should it be monitoring, when exactly should it be monitoring that, when should it transfer the data from local to azure storage and so on. Alright, so lets take a look at how this configuration thing works.

1. Diagnostic Monitors have a default configuration. This configuration enables you to directly deploy your Windows Azure Application without typically having to take any extra efforts for setting up the DM. The only care that you have to take is ensure that the DM is initialized in your role (more later) and that a storage credential has been provided for logging the traces. Here’s a code-snippet that retrieves the default configuration of a Diagnostics Monitor:

public class WebRole : RoleEntryPoint
    {
        public override bool OnStart()
        {
            // For information on handling configuration changes
            // see the MSDN topic at http://go.microsoft.com/fwlink/?LinkId=166357.

            DiagnosticMonitorConfiguration diagConfig = DiagnosticMonitor.GetDefaultInitialConfiguration();


            return base.OnStart();
        }
    }

The DiagnosticMonitorConfiguration is a class that holds all the configurable properties of the DM. The GetDefaultInitialConfiguration method of the DM returns the default configuration that the DM is currently equipped with. Lets see what this Default Configuration contains.

Ahh. Now we know what the DM has that we can configure. Big deal. The ConfigurationChangePollInterval property is set to 1 minute. This means that the DM will look for a change in its configuration every one minute. But where does it look? Patience my love.  There are other properties that can be set and all these properties are nothing but elements present in the DiagnosticMonitorConfiguration schema. I will not go in depth about these properties in this blog post simply because I myself am yet to further explore them but rest assured my future posts will attempt to reveal the mysteries behind these.

2. Moving on. Once you have obtained the default configuration you can joyfully modify the configuration settings described above. Note that the code has been added within the OnStart method of your role. This means that you are altering the DM settings before the role is starting. Here’s an example wherein I have configured the DM to transfer only those Infrastructure Logs which have been logged in occurrence of an error. In addition I have also configured the transfer of the logs between local and azure storage to initiate every 30 seconds. Once the configuration is done I start the Diagnostics Monitor by providing it with these initial configs.

public class WebRole : RoleEntryPoint
    {
        public override bool OnStart()
        {
            DiagnosticMonitorConfiguration diagConfig = DiagnosticMonitor.GetDefaultInitialConfiguration();

            diagConfig.DiagnosticInfrastructureLogs.ScheduledTransferLogLevelFilter = LogLevel.Error;
            diagConfig.DiagnosticInfrastructureLogs.ScheduledTransferPeriod = new TimeSpan(0, 0, 30);

            DiagnosticMonitor.Start("Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString", diagConfig);

            return base.OnStart();
        }
    }

The Start method of the DM takes in two parameters here. One is the DiagnosticStorageAccountConfigurationSetting Name. This is the name of the setting that you have provided in your cscfg for the DiagnosticsConnectionString. This will be further clear once we move to the DM Initialization part. The other parameter is the DiagnosticMonitorConfiguration object that we have just initialized and modified to suit our fancies. 

The Start method can also be invoked by passing only the ConfigurationSettingName mentioned above in which case the DM will be started using the default config as explained in the first point. The other set of parameters that the Start method can take instead are the CloudStorageAccount object itself and then our DiagnosticMonitorConfiguration object (diagConfig).

3. So that was all about configuring the DM before the role starts – in short we set up a start-up task for the role. What if there are some configs that must be applied at runtime?Say that your application contains a Web and a Worker Role. After a particular event in the application – say an Administrator login – you need to monitor a Performance Counter of the Worker Role to check on the health of the VM. Oops! But you haven't configured the DM to log that particular information. Ta-Da. Presenting the RoleInstanceDiagnosticManager. This little superhero enables you to configure the diagnostics of a single role instance. Here’s how it works:

RoleInstanceDiagnosticManager instanceDiagManager = new RoleInstanceDiagnosticManager
                (CloudStorageAccount.Parse("Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString"), 
                "<deployment-id>", 
                "<role-name>", 
                "<role-instance-id>");

DiagnosticMonitorConfiguration diagConfig = instanceDiagManager.GetCurrentConfiguration();

So the RoleInstanceDiagnosticManager is initialized using 4 parameters : the CloudStorageAccount, the deploymentID, the Role Name, the RoleInstanceID, all of which can be retrieved using ServiceManagementAPI’s. Once initialized it can be used to obtain the current DiagnosticMonitorConfiguration using GetCurrentConfiguration. Note that the earlier config that we dealt with was the default one and this config is the one that the role would be equipped with at that moment of time. And that’s it. You can then use the diagConfig to alter the configuration settings of the DM (like in the startup task) and then call the instanceDiagManager.SetCurrentConfiguration(diagConfig) to set the diagnostics configurations of the DM for that particular Worker Role.

4. Another way to go about configuring the DM would be the PowerShell way. If you are into Windows Azure you have to be a PowerShell guy. Being the ‘suh-weeeet’ topic that it is, it deserves a blog post of its own and I being a worthy disciple of GeeksAndTheirInternet , will comply.  My future blog posts will boast about the ways in which PowerShell can be used to remotely configure diagnostics. That’s right – remotely.

For now, lets get back to the DM initialization part. The what? Now, if you did scroll up and eventually notice that the 1st point in the DM config part did mention this vaguely, you would at this moment be going “Uh-huh. The DM initialization. ‘Course I remember”. Now, the DM initialization isnt that big a deal. In fact, chances are that in case you launched your first win azure app you didn’t even realize that you had initialized the DM. That’s because Visual Studio 2010 provides the Cloud solution with in-built csdef and cscfg’s that already have a Diagnostic Setting enabled for each role. Don’t believe me? Open your cscfg for a new Azure project and I bet what you find is this:

<ConfigurationSettings>
  <Setting name="Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString" value="UseDevelopmentStorage=true" />
</ConfigurationSettings>

The settings described herein are the ones that provide the Storage Credentials of your Windows Azure Storage Account. While pushing your app on to the cloud it is imperative that you provide your storage credentials here. The DM will then know where to transfer the logs from the local storage. This setting name is the same that we had used as a parameter in the Start method of the DM in point 2 of our DM Configuration part. Thus the DM will use the storage credentials provided here to perform its activities.

The csdef is also built to import the Diagnostics Module.

<Imports>
    <Import moduleName="Diagnostics" />
</Imports>

And if you open your ASP.NET Web Role’s web.config, you will very likely find this:

<system.diagnostics>
  <trace>
   <listeners>
     <add type="Microsoft.WindowsAzure.Diagnostics.DiagnosticMonitorTraceListener, Microsoft.WindowsAzure.Diagnostics, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" name="AzureDiagnostics">
       <filter type="" />
     </add>
   </listeners>
  </trace>
</system.diagnostics>

The above code is nothing but the declaration of a trace listener within the Web Role. This trace listener can then be used to trace errors within the application. Example:

try
{
  string s = null;
  if (s.Length > 0)
  {
    // Hmm...Something tells me that this is not going to work! 
  }
}
catch (Exception ex)
{
  //Well.. at least I was right about something
  System.Diagnostics.Trace.TraceError(System.DateTime.Now + " Error: " + ex.Message);
}

The System.Diagnostics.Trace.TraceError method will now log the error into WAD Logs since the trace listener is in place.

Thus the DM has been initialized using nothing but simple Storage Account Credentials. Another question here is how and when does the DM know that the configuration has been changed? Well, remember that itsy-bitsy configuration setting – ConfigurationChangePollInterval – that we saw earlier. That setting decides the time interval at which the DM polls the azure storage to check if there are any changes that have been made to its configuration. Uhhh. Ok. But why does the DM poll the azure storage? ‘Cos that’s where the diagnostic configuration setting file has been kept silly! Wait. I dint tell you about the Diagnostic Configuration setting file? Oops! Remember when we talked about DiagnosticMonitorConfiguration schema earlier in the 1st point on DM config. Windows Azure maintains an XML file (extension as *.wadcfg) similar to your cscfg which is the grand summation of all the settings of your DM. This file is typically present in your Wad-Control-Container blob container within your Win Azure Blob Storage and also in your local storage within the root directory of your role. Here is a comprehensive detail of the schema. So that’s why the DM keeps polling the azure storage (blob) to check for any changes in the configuration and apply them accordingly.

Well, that just about covers the basics of Windows Azure Diagnostics. As promised, posts ahead will definitely dare to tread the fields of Diagnostic Configuration Settings elaboration and PowerShell usage to configure diagnostics.

Happy coding