BizTalk and RabbitMQ

The same article in the TechNet.

If you are working with queues in BizTalk Server, the most possible it is the MSMQ (in my experience).

MSMQ is an old man of the Microsoft technology stack. It was created when there were no good standards for messaging. Now MSMQ is wrapped partly in the .NET System.Messaging namespace. It is just a small facelift. MSMQ is still a proprietary technology without well-defined messaging protocol. It means, you cannot use the MSMQ messaging protocol without MSMQ itself.

Now we see vortex of new messaging standards and technologies. One on the top is the AMQP standard and one of the bold implementation of AMQP is the RabbitMQ.

It is one more protocol, one more messaging system which, for sure, can be integrated with BizTalk Server.

Here I will implement the standard queue messaging: sending and receiving messages from BizTalk down through the RabbitMQ queue. I am using the WCF-Custom LOB Adapter (provided from BizTalk side) and the WCF RabbitMQ Binding (provided from RabbitMQ side) to link two systems.

Installation and preparation

Assuming the BizTalk Server and the Visual Studio 2010 are also installed.

Installing RabbitMQ Service

  1. Go to the RabbitMQ site and download the last versions of server and client for Windows. For me it was rabbitmq-server-2.7.1.exe and rabbitmq-dotnet-client-2.7.1-dotnet-3.0.zip. Client includes the WCF binding which I will use lately. There are several good manuals: rabbitmq-dotnet-client-2.7.1-user-guide.pdf and rabbitmq-dotnet-client-2.7.1-wcf-service-model.pdf. I was quite impressed by quality distributives and documentation.
  2. Start rabbitmq--server-x.x.x.exe. It will request to install Erlang. Agree, go to the Erlang site [http://www.erlang.org/] and download it. For me it was Erlang.otp_win32_R15B.exe
  3. Start Erlang…exe. Installation went smoothly.
  4. Start rabbitmq--server-x.x.x.exe again. Installation went smoothly.

RabbitMQ service is installed. You can see it in the Services window.

Creating RabbitMQ Client Base Assemblies

 Now it is time to install the client for .NET and WCF binding extension. Use this manual.

  1.  Downloaded the .msi. Installed it. Hmm… no Rabbit.Client.dll (When I started with Examples, they all need this file. It is a main client dll).

  2. Downloaded the *-dotnet-3.0.zip. Nope, it was wrong.
  3. We need the last rabbitmq-dotnet-client-2.7.1.zip file. Extract everything from this zip.
  4. Made a copy of the Local.props.example file to Local.props file in the same root folder.
  5. Start Visual Studio with RabbitMQDotNetClient.sln and successfully build it. The ..\rabbitmq-dotnet-client-2.7.1\projects\wcf\RabbitMQ.ServiceModel\obj\Debug\RabbitMQ.ServiceModel.dll was created. This dll together with the RabbitMQ.Client.dll are the assemblies used by the RabbitMQ clients.

Set up the RabbitMQ binding extension

Here I really lost a lot of time. It is easy for the C# clients; the RabbitMQ.ServiceModel.Test gives all the information. But we have to use this binding from the WCF-Custom LOB adapter. To do so we have to add the assemblies to GAC, change machine.config, etc. See the WCF LOP Adapter documentation.

  1. Sign the RabbitMQ.ServiceModel and RabbitMQ.Client projects and GAC-ed them.
  2. Insert  an <add> node into the config section of machine.config:

<extensions>
      <bindingExtensions>
        …
        <add  name="RabbitMQBinding" 
          type="RabbitMQ.ServiceModel.RabbitMQBindingSection, RabbitMQ.ServiceModel, Version=0.0.0.0, Culture=neutral,   PublicKeyToken=1751e286f1ab778d"/>
      </bindingExtensions>
    </extensions>

Note:

  • When the binding extension assembly is placed in GAC, we cannot use the above section in the app.config file! This <add> element should be placed only into the machine.config file. Otherwise it cannot be seen in the binding extension list of the WCF LOB Adapter.
  • Use >Gacutil.exe /l RabbitMQ.ServiceModel
    to get the real parameters for your assembly.
I have several machine.config files in folders:
    • C:\Windows\Microsoft.NET\Framework\v2.0.50727\CONFIG
    • C:\Windows\Microsoft.NET\Framework\v4.0.30319\Config
    • C:\Windows\Microsoft.NET\Framework64\v2.0.50727\CONFIG
    • C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Config

Which one to use? My VM is 64bit. BizTalk Host of the port is 32bit. BizTalk is the 2010 version. In my case the right file was the C:\Windows\Microsoft.NET\Framework\v4.0.30319\Config\machine.config.



Finally the RabbitMQ binding installed and we can see it in the Binding Type list for WCF-Custom adapter.

Testing

Creating the BizTalk test

I have created two pairs of port for testing.

File RP -> WCF-Custom (RabbitMQ) SP ->

 [RabbitMQ Server]

 -> WCF-Custom (RabbitMQ RP) -> File RP

 

I’ve changed only two parameters: “hostname” and “oneWay” and let others be default:

 

URL-s for the Send and Receive RabbitMQ Ports set up as “soap.amqp://192.168.11.128/Mytest” for both of them.

Let’s test it.

Small text file was dropped into In folder, consumed by Receive Port, went through the RabbitMQ queue, and appeared in Out folder. Success.

Note:

  • Now the RabbitMQ binding is set up to process only well-formed Xml documents.
  • “localhost” as “hostname” works fine.

Using the RabbitMQ Binding Element

The RabbitMQ binding is limited to the Text messaging encoding. If we need to change it or add more binding extensions, we have to use the customBinding type and use the rabbitMQBindingElement together with another element (including another encoding). How to do this?

1. Add the <add> node to the machine.config:

<bindingElementExtensions>
                           …
        <add name="rabbitMQBindingElement" type="RabbitMQ.ServiceModel.RabbitMQTransportElement, RabbitMQ.ServiceModel,   Version=0.0.0.0, Culture=neutral, PublicKeyToken=1751e286f1ab778d"/>
</bindingElementExtensions>

 2. Change the Receive Location transport parameters. It was the RabbitMQBinding. Change it to the customBinding. The Transport Binding Element has to be changed to the rabbitMQBindingElement. I changed only one parameter: hostname as “localhost”.

 

 3. Test it as previously.

Conclusion

Using RabbitMQ as one more transport for BizTalk and/or WCF applications is possible and straightforward. The learning curve is short. There are not too many issues with setting up the RabbitMQ Service and Client parts. Actually there wasn’t any RabbitMQ part which was exposed to the BizTalk directly. RabbitMQ works only with WCF-Custom adapter and only this adapter communicates with RabbitMQ Service.
Now the RabbitMQ implements limited client functionality for the WCF. RabbitMQ implements the AMQP v.0.9 protocol, not the last version 1.0.

Next steps

Next steps are to test this solution with different messaging patterns and try to figure out how to use RabbitMQ advantages (if there are some) over the MSMQ. Potentially the advantages are the simplest implementation of the Request/Response and Duplex patterns, better scalability, better performance, better manageability and support.

 

Print | posted on Thursday, February 9, 2012 1:18 PM

Feedback

# re: BizTalk and RabbitMQ

left by Wayne at 2/9/2012 3:48 PM Gravatar
You say, "If you are working with queues in BizTalk Server, the most possible it is the MSMQ."

Not so -- you can also access JMS queues through the JNBridge JMS adapter for BizTalk Server. http://www.jnbridge.com/JMS-BizTalk-Adapter.htm

# re: BizTalk and RabbitMQ

left by Leonid Ganeline at 2/9/2012 4:02 PM Gravatar
Sure, I will add "in my experience". Thanks for note!

# re: BizTalk and RabbitMQ

left by Tord Glad Nordahl at 2/10/2012 12:01 AM Gravatar
Nice one Leonid! good explaining!

# re: BizTalk and RabbitMQ

left by Leonid Ganeline at 2/10/2012 7:55 AM Gravatar
Thank you Tord!

# re: BizTalk and RabbitMQ

left by mutuelle at 2/15/2012 11:24 PM Gravatar
Nice to read your blog posts here,one day I want to install it,but,I missed that Net Framework,It took time for me to fix this

# re: BizTalk and RabbitMQ

left by Francois Malgreve at 9/15/2012 3:08 PM Gravatar
Great post Leonid! At the end of your post, you are questioning the advantage of using RabbitMQ over MSMQ. Would it not simply be that it could be a great way to integrate BizTalk with non-Windows platforms in intranet scenarios? Even in intranets, the common approach is HTTP web services but I could imagine scenarios in which a durable queuing mechanism would be a better approach than standard web services. RabbitMQ seems very interesting actually, thanks for sharing!

# re: BizTalk and RabbitMQ

left by Leonid Ganeline at 9/15/2012 6:30 PM Gravatar
Thank you Francois.
Yes, I also think the queues are good complimentary technology to the web-services.

# re: BizTalk and RabbitMQ

left by Linesh Gajera at 10/1/2012 12:33 PM Gravatar
Leonid

I followed the article exactly you mentioned but somehow I am getting following error on Receive Port.

I am also not able to send messages to queue as well. Though there were no error on Send port but somehow message does not show up in the queue.

Does the queue have to be Durable/Transient? I am trying to send to already existing queue

The Messaging Engine failed to add a receive location "RabbitMQReceiveLocation" with URL "soap.amqp://localhost/MyTest" to the adapter "WCF-Custom". Reason: "System.InvalidOperationException: No broker was specified.
at RabbitMQ.ServiceModel.RabbitMQTransportBindingElement.BuildChannelListener[TChannel](BindingContext context)
at System.ServiceModel.Channels.Binding.BuildChannelListener[TChannel](Uri listenUriBaseAddress, String listenUriRelativeAddress, ListenUriMode listenUriMode, BindingParameterCollection parameters)
at System.ServiceModel.Description.DispatcherBuilder.MaybeCreateListener(Boolean actuallyCreate, Type[] supportedChannels, Binding binding, BindingParameterCollection parameters, Uri listenUriBaseAddress, String listenUriRelativeAddress, ListenUriMode listenUriMode, ServiceThrottle throttle, IChannelListener& result, Boolean supportContextSession)
at System.ServiceModel.Description.DispatcherBuilder.BuildChannelListener(StuffPerListenUriInfo stuff, ServiceHostBase serviceHost, Uri listenUri, ListenUriMode listenUriMode, Boolean supportContextSession, IChannelListener& result)
at System.ServiceModel.Description.DispatcherBuilder.InitializeServiceHost(ServiceDescription description, ServiceHostBase serviceHost)
at System.ServiceModel.ServiceHostBase.InitializeRuntime()
at Microsoft.BizTalk.Adapter.Wcf.Runtime.BtsServiceHostBase.InitializeRuntime()
at System.ServiceModel.ServiceHostBase.OnOpen(TimeSpan timeout)
at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)
at Microsoft.BizTalk.Adapter.Wcf.Runtime.WcfReceiveEndpoint.Enable()
at Microsoft.BizTalk.Adapter.Wcf.Runtime.WcfReceiveEndpoint..ctor(BizTalkEndpointContext endpointContext, IBTTransportProxy transportProxy, ControlledTermination control)
at Microsoft.BizTalk.Adapter.Wcf.Runtime.WcfReceiver`2.AddReceiveEndpoint(String url, IPropertyBag adapterConfig, IPropertyBag bizTalkConfig)".

# re: BizTalk and RabbitMQ

left by biztalk techie at 1/28/2013 5:06 PM Gravatar
I was trying to build the aplication using the article . I manage to successfully creat ana integration application.I want to ask is is possible to have a message onto a queue without a soap header. When my application sends a message on the queue, it is encapsulated in a soap envelope as below :
<soap :envelope>
<header>...... </header>
<body> payload(actual xml message) </body>
</soap: enevelope>

# re: BizTalk and RabbitMQ

left by Leonid Ganelin at 1/29/2013 9:16 AM Gravatar
@biztalk_techie: try to use the Transport Properties window of the WCF-Custom adapter, the Message tab. Here you can choose which part of the message would be used as a body.

# re: BizTalk and RabbitMQ

left by biztalk_techie at 3/5/2013 7:32 AM Gravatar
@ Leonid -Thanks for your comments. I want to publish messages to queue without being encapsulated into soap envelipe and also at the same time i want to subscribe to messages deom mq which are not encapsulated within the soap envelope. I have tried alomost everything availble at port configuration esp. messages tab configuration. It does not helped me. Did you tried using these configurations?

# re: BizTalk and RabbitMQ

left by Linesh Gajera at 5/20/2013 7:16 AM Gravatar
Nice Article Leo,

I tried using WCF RabbitMQ Adapter in BizTalk but unfortunately, it really fall short of expectations. It seems there is a serious issue with it.

1) When you stop the receive location, it actually kills the Host Instance with following error.
Faulting application BTSNTSvc.exe, version 3.8.368.0, time stamp 0x49b1dadf, faulting module kernel32.dll, version 6.0.6002.18449, time stamp 0x4da47a32, exception code 0xe0434f4d, fault offset 0x0001c83b, process id 0x1c44, application start time 0x01ce55638720c8be

2) Second, let's say If I need to restart Rabbit MQ Server, it kills the BizTalk server host instances and Receive location stop/start does not work until you restart the host instance.

Somehow, the error handling and recovery is not graceful in RabbitMQ WCF Adapter for BizTalk.

We heavily use RabbitMQ but WCF Adapter is just not there yet.

I do not know others have faced the same issues

Thanks

Linesh

# re: BizTalk and RabbitMQ

left by Patrick Smith at 3/18/2014 9:23 AM Gravatar
Hi Leo.

Thanks for a great post. I followed your article exactly as stated above but when I submit a message to BizTalk, the message is consumed bgy the Receive location and just disappears. It does not get routed to the RabbitMQ queue but also it does not throw any exceptions, log any events or even appear in any activity queries in the BTS Admin console. Submitting to and consuming from the same RabbitMQ queue via a console app works, so I guess the queue is set up correctly. Can you tell me if there is any additional configuration needed to get BizTalk to submit the message to RabbitMQ? Or if I need to bind the q to an exchange and reference the exchange in the send port uri ie. soap.amqp://localhost/myExchange. Or should I be referencing the queue by name here. ie. soap.amqp://localhost/myQueue.

Any help would be greatly appreciated.

Thanks

Patrick
Post A Comment
Title:
Name:
Email:
Comment:
Verification: