As programmers, we can build bridges between various systems in two ways: point to point, or loosely coupled. Within each design paradigm, additional options are available to us, such as which tools and platforms we can use, and how we actually perform the integration, such as whether or not we need to split large workloads into smaller chunks. Generally speaking, the methods we choose to use to perform the work (also known as integration patterns) can be implemented using most technologies; however some technologies make it easier to perform certain tasks than others. To this end, let’s take a look at the Service Bus for Windows Server (that I will call simply Service Bus going forward), a platform allowing loosely coupled integrations, and some of its major features.
About the Service Bus
The Service Bus is a messaging technology that you can install and configure on your own virtual machines. You can think of the Service Bus as a messaging technology that sits between MSMQ and BizTalk. It is similar to MSMQ in the sense that it deals with messages, and it is similar to BizTalk in the sense that it is a Publication/Subscription framework for distributing workloads and processes. Of course, BizTalk is a full blown integration platform which allows you to design integration workflows, which is not the case with the Service Bus by itself. However, there was a technology gap between MSMQ and BizTalk, and the Service Bus fills it. Its first incarnation was in Microsoft Azure, as the Azure Service Bus. The on-premises version is called the Service Bus for Windows Server, and can be configured using PowerShell scripts, or a user interface highly similar to the Microsoft Azure platform by installing the Windows Azure Pack. In order to install and manage your Service Bus with the Windows Azure Pack, I recommend you first install the Azure Pack, create a Tenant and a Subscription, and then the Service Bus using the Configuration Wizard (where you will be able to specify the tenant account you created earlier). This will make it easier to configure your Service Bus later.
In the world of the Service Bus, a Topic is a receiving channel for messages, and Subscriptions are outgoing channels that programs read messages from. When you configure a Topic, you usually configure at least one Subscription. A program sends messages into a Topic, which are moved into the Subscriptions defined, and other programs read the messages from the Subscription. In other words, a Subscription is also a queue.
The Service Bus comes with very interesting capabilities, some of which are detailed below. At a high level, you can use the Service Bus in two ways: either as a messaging platform (much like MSMQ), or as a Pub/Sub implementation in which one queue can forward messages to other queues (the key foundation of a service bus). The first one is called Queues, and the second Topics. You can think of a Topic as a more flexible queue, because you can perform certain key tasks without coding. Let’s review the following capabilities that can be performed with Topics: Routing, Filtering, an Dead-Letter Queue. For a more in-depth overview of Queues, Topics, and Subscriptions, take a look at this MSDN article.
Routing is the ability to forward a message for consumption into one or more queues. The key concept is that there could be multiple destination queues (Subscriptions); if a Topic has 5 subscriptions, the message will be routed to all 5 subscriptions, for 5 applications to read from. Let’s say you are dealing with a system that enrolls students for classes; the system responsible for registering classes will send a single message into the Enrollment Topic to indicate that an enrollment has completed. Assuming 2 systems are interested in the Enrollment topic, such as the Financial Aid and the Student Housing systems, you would create two Subscriptions (one for each). This achieves loose coupling because the Registration module has no idea how many systems, if any, will be interested in receiving enrollment completion events.
Filtering is the ability to select which Subscription will receive the message sent to a Topic. To follow our previous example, we could say that the Student Housing system wants all messages (in which case no filter is defined), but the Financial Aid system is only interested in messages for which at least 2 classes were selected. You could create a filter on the queue by specifying that a custom property on the message, which contains the count of selected classes, must have a value of 2 or higher. This allows you to reduce the workload on the Financial system by filtering out messages that do not meet specific criteria.
A deal-letter queue is a system queue in which problem messages are stored. For example, if a message creates downstream issues, such as a system crash, it may never be removed from the queue properly by the consuming application. An internal counter specifies how many times a message has been accessed by consumers, and if it has been accessed too many times, the Service Bus will assume that the message has a problem, and will be moved into the Dead-Letter queue automatically. This allows the Service Bus to remove poison messages that affect the stability of the bus.
Implementing loosely couple systems can have significant benefits for organizations that need to orchestrate multiple systems efficiently, without hard-coding their integration needs. The Service Bus has many more features, but these are the ones that caught my attention. I encourage you to learn more about the Service Bus for Microsoft here.
About Herve Roggero
Herve Roggero, Microsoft Azure MVP, @hroggero, is the founder of Blue Syntax Consulting (http://www.bluesyntaxconsulting.com). Herve’s experience includes software development, architecture, database administration and senior management with both global corporations and startup companies. Herve holds multiple certifications, including an MCDBA, MCSE, MCSD. He also holds a Master’s degree in Business Administration from Indiana University. Herve is the co-author of “PRO SQL Azure” and “PRO SQL Server 2012 Practices” from Apress, a PluralSight author, and runs the Azure Florida Association.