BizTalk: the Naming Conventions in Examples

See the BizTalk: BizTalk Solution Naming Conventions article.

In small application we do not really care about names. When the number of objects start growing, we start pay attention to the names. Experienced developers working with big projects are recognizable by the carefully crafted names in the developed code.

When a new developer start working in a new team, he/she spent first several hours to read the general level documentation and the naming convention usually is a part of it. Then he/she start to develop and now time should be spent to review the existed code base to become accustomed to the coding standards, and again the naming conventions are the main parts of them.

So the naming conventions document are important but the existed code base is also important. The documentation could be not up to date, but the code always is.

The code example easily could replace the the documentation, that’s why I decided to show the BizTalk naming convention in the real-life examples.

Example description

This BizTalk deployment is compounded from many BizTalk applications. Each of application has 1-10 Visual Studio projects. The applications were developed through the many years by different development teams. Applications integrate several systems. Some systems have just one interface, some systems have many interfaces. In most cases one system interface is integrated with another system interface. Sometimes several systems are integrated with one system interface.

One solution is chosen for this example. Why the GLG.Samples.Name.Shared.Schemas solution was created in real life?

As we know an interface in BizTalk defined by a protocol and one or several XML schemas.  In many cases those schemas are dictated by the integrated system not by the BizTalk application developer. Usually the BizTalk developer uses the Adapter Wizard to generate/import these schemas. These schemas are managed (created, modified) not by BizTalk application developer but by the external system owners. Let’s call these schemas the external schemas.

One interesting aspect of the external schemas is one schema can be used by several BizTalk applications. As we know the schemas for the receive locations should be unique, they cannot be deployed in several assemblies. That means if we want to use the same schema in several projects we should share an assembly with this schema, reference this assembly from all these projects.

In this situation the special attention is paid to these schemas.

All external schemas are placed in a separate Visual Studio solution. The schemas are grouped by the systems. Inside each system the schemas are grouped by the interface names.

This design gives us one more useful feature. Here we always know where to find the external schemas. They are not spread through many projects and application but always placed and can be found in one solution.

The sample provide two versions of the naming conventions. I called them "Long Names" and "Short Names".

Long Names

It is the Solution Explorer view:

Solution.Long

It is the folder view:

Folders.Long

Short Names

It is the Solution Explorer view:

Solution

It is the folder view.

Folders

Comparing

The Short Name convention looks more clean. There are no generic prefixes (GLD.Samples.Names.Shared.Schemas) inside the Project and folder names. If you fight the well-known issue with the TFS limitation on the size of the full file names, it would be a preferable variant.

I personally prefer the Long Name conventions. The project name is equal to the project namespace and the project assembly name. The full project names are better in situations where a project might be included in several Visual Studio solutions or the projects could be frequently shuffled between solutions.

Q&A :

  1. Is it OK to use the names compounded from 4 and more pieces like GLD.Samples.Names.Shared.Schemas?
  2. Do we need the GLD.Samples prefix in the BizTalk application name, in the solution name?
  3. Why we use the _Ops suffix for the SystemA, SystemB, and SystemC projects?

BizTalk Applications

Here is the BizTalk Administration Console view:

BizTalkAdminConsole

As you can see the GLD.Samples.Names.Shared.Schemas solution is deployed as the Names.Shared.Schemas application. The GLD.Samples prefix is removed because all deployed solutions are using this GLD.Sample prefix. If all artifacts has the same name part, this part definitely can be removed. Why this part is not removed from the solution/project name? Because projects (assemblies) are working together with many system, Microsoft and others assemblies in one global name space, in .NET run-time space. All assemblies should be placed in GAC, where GLD.Samples prefix helps us to find out our assembles. This is an answer on the second question. Smile

This is the Short Name convention for the application names.

The Long Name convention is simple:

 the BizTalk Application Name equal the Visual Studio Solution Name

I personally prefer the Long Name convention as a faster way to work, copy/past without any change, and do not guess the full name of the assemblies.

How complex could be the name hierarchy?

For example, we have such name as GLD.Samples.Names.Shared.Schemas.Sap.AV_CREDIT_STATUS_MODIFY. Isn't it too complex?

Not at all. All parts of this name are here for a reason. Each part shows the grouping of an object. Our names unambiguous show us the object hierarchy. Just try to avoid some parts to get something shorter and you would be in trouble. You would create not one but numerous naming conventions and numerous rules how to use them.

Hierarchical naming conventions is the king of the BizTalk projects. These projects are complex enough to create the object grouping hence hierarchy. But they are not too complex to push us in the areas, where the hierarchical grouping is not working well, where keys or tags are working better.

I hope this is an answer for the first question.

Now the answer for the third question. Why we use the _Ops suffix for the SystemA, SystemB, and SystemC projects?

For instance, we have GLD.Samples.Names.Shared.Schemas.SystemA.CreditCheck_Ops project
why it is not just
GLD.Samples.Names.Shared.Schemas.SystemA.CreditCheck ?

The problem is, this project has a schema which has several roots and one of them is the CreditCheck. What is wrong with it, why we cannot use the same name for a part of the project name and for a schema root?

The XML schema is serialized into the .NET classes, and a schema root serializes into one such class. So we will get a class with full qualified name as GLD.Samples.Names.Shared.Schemas.SystemA.CreditCheck.ChreditCheck. And while the project is building, we got an error like this:

Error.SymbolXIsAlreadyDefined

Description would be a little bit different from the picture about: “symbol ‘GLD.Samples.Names.Shared.Schemas.SystemA.CreditCheck’ is already defined; the first definition is in assembly…”.

For some reason the .NET builder parses the names sometimes left-to-right, sometimes right-to-left and this creates this kind of errors.

To avoid this kind of error we have to avoid creating .NET classes with name equal one of the namespace part. The "_Opr" suffix is used for this purpose.

This is an implementation detail, but we have to take care of it in the naming conventions.

Conclusion

There is no single naming convention for all and every situation. Be reasonable and do not create an over engineered conventions. They are not the goal in development but the helpers.

Quiz for readers Smile

  • Propose your improvements to these naming conventions.
  • What are the Pros and Cons of placing the external schemas in a separate solution?
  • This solution could be very big. Would be this a problem?

BizTalk: Internals: Schema Uniqueness Rule

A source code could be downloaded from here.

Sometimes in BizTalk the artifact visibility can be global. We place the artifacts in different assemblies and it doesn’t change visibility. We place the artifacts in different BizTalk applications and it doesn’t change visibility. 
Let’s talk about the schemas. In my previous post BizTalk: Internals: namespaces I’ve shown that schemas have additional parameter, the Xml [target] namespace. Why it is so important?
The BizTalk receives messages on the Receive Locations. Theoretically messages are just the byte arrays. But the BizTalk frequently has to do additional tasks like changing the message format from the one system format to another. That means the message should be received in an known format. Without knowing the format the BizTalk cannot make a transformation. The BizTalk message internal format is an Xml or a byte array. Here I am talking about the messages in the Xml format.
So the Xml messages are received by the Receive port as the byte arrays and should be parsed into Xml format. The XmlReceive pipeline makes this transformation. The first thing the XmlReceive makes, it searches a Xml namespace and a root node. Those two parameters create a MessageType parameter with is promoted as a context parameter.
As you can see the BizTalk uses the MessageType to find out the schema for this message. This Xml schema is used to parse the whole message into Xml format.
So the first step is to find a Xml namespace and a root node inside the byte array regardless of anything else, the second step is to find out the Xml schema, then this schema is used to parse the whole message into Xml format.
Now we are discussing the second step, how the BizTalk is searching the right schema.
The BizTalk searches through the whole schema list regardless of the application boundaries. Each schema belongs to one of the BizTalk application, but BizTalk ignores this. The global schema list is stored inside the BizTalk management database. When we deploy a project with a schema, the schema is registered inside this database.
When an inbound  message is received and processed by the XmlReceive pipeline, the BizTalk extracts a MessageType (a namespace + a root node), and searches for this MessageType in the management database. An error is raised if the MessageType is not found. The error is raised if more than one schemas with this MessageType is found.

An important note is the schema uniqueness rule is not verified at the deployment time.
I have created several samples to show how it works. These samples are barebone projects, focused on the “schema uniqueness” rule.

Identical schemas in the one BizTalk application

The application includes two projects. Each project is compounded from one schema.

OneApp.VSSolution

Both schemas have the same structure and the same Xml namespaces. Schema1

The only difference is the .NET namespaces. Each schema is placed in different assembly. OneApp.DeployedSchemas.

I have created one receive port with one File receive location and XMLReceive pipeline. A send port works as a subscriber to this receive port.

Nothing else is in this application. Two identical schemas in different assemblies and receive and send ports.

I droped a test Xml file in the receive folder. What do you think is happened? You are right, I’ve got an error and two related events, 5753 and 5719. OneApp.EventsOneApp.Events.5753OneApp.Events.5719

It is a famous "Cannot locate document specification because multiple schemas matched the message type ###" error. :)

Conclusion: schemas of the receive messages should be unique across allne BizTalk application.

Identical schemas in the different BizTalk applications

Now two the same projects are in different BizTalk applications. (As you can see I have broken the naming convention, which forces to place the BizTalk applications in the different Visual Studio soulutions. In real life there would be two Visual Studio solutions. Sorry for that. Smile)

VSSolutionFirstApp.Schema1SecondApp.Schema1

The ports are the same as they are in the first sample.

When I droped a test file I have got the same error as in the first sample. So the schemas are now placed in different applications but it doesn’t change the schema visibility. The schema from one application is visible from another application.

Conclusion: schemas of the receive messages should be unique in different BizTalk applications. In other words they should be unique across the whole BizTalk Server Group.

Identical schemas imported by different schemas

Now let’s ask a question. Is the “schema uniqueness” rule relating to the imported schemas?

Here is a real-life example. We are consuming the WCF web-service metadata, and it creates the schema set where we can see the Microsoft Serialization schema. It is the same schema almost in each and every WCF service. Do we have to take care of this schema if we consume several WCF services? Do we have to “fight” this schema, extract it into Schared project, etc.?

This sample is the same as a previous sample with two additional schemas: Importing.xsd and Imported.xsd.
VSSolution 

Both Importing schemas in two applications are different. They have different Xml namespaces.
ImportingSchema

But both Imorting schemas are identical.ImportedSchema

The ports are the same as they were in the first sample.

When I droped a test file there was no error.

So the “root” schema namespaces, the namespaces which define the MessageType were different, but the namespaces of the imported schemas were the same and this the XMLReceive pipeline could successfully recognize the MessageType of the message.

Conclusion: the “schema uniqueness” rule works only for the schema which define a MessgeType of the received message and doesn’t work for the imported schemas.

Notes:

  • Two parameters of Imported schema are changed from default values. It forces to include the namespace prefixes of the Imported schema into the Xml documents. Those parameters are the AttributeFormDefault and ElementFormDefault.

    ImportedQualifiedParam

    A message with these parameter values equal “Qualified”:Xml.Qualified

    A message with these parameter values equal “Default”:Xml.Unqualified
  • We could mention the “typeOnlySchema” Root Name for the Imported schema. Actually this schema doesn’t have any root, any elements, it includes only types.
    TypeOnlySchema

“Schema Uniqueness” Rule

The schemas used for the MessageType should be unique within the BizTalk Service group if these schemas are used in the XMLReceive pipeline of the receive locations.

Is there a reason for the “Schema Uniqueness” rule?

Is there a good reason for this rule?

I would prefer if the schema visibility is limited inside the BizTalk application. It would extrimply simplify the schema and service versioning.

Let’s take an example when our partner added a new version of the service. The service schemas itself are different in both services (several nodes were removed, several nodes were added), but the schema namaspaces and roots are the same in both services. The new and old services work side-by-side and we have to consume both service versions. If we could limit the schema visibility inside the application boundaries, creating the BizTalk applicatons for consuming both service versions would be simple. Now it is cumpersome. I am sure, each experienced BizTalk deverloper can remember several real-life examples, when this “schema uniqueness” rule was a reason when several hours project was instead implemented in several days.

Looks like a reason for this “schema uniquness” rule is simple. The BizTalk application was introduced only in the BizTalk 2006. In this time Microsoft decided do not change the XMLReceive pipeline to embrace new application conception and since then this feature was not on the priority list in the BizTalk development.

BizTalk: Internals: the Partner Direct Ports and the Orchestration Chains

Partner Direct Port is one of the BizTalk hidden gems. It opens simple ways to the several messaging patterns.

This article based on the Kevin Lam’s blog article. The article is pretty detailed but it still leaves several unclear pieces. So I have created a sample and will show how it works from different perspectives.

Requirements

We should create an orchestration chain where the messages should be routed from the first stage to the second stage. The messages should not be modified. All messages has the same message type.

Common artifacts

Source code can be downloaded here.

It is interesting but all orchestrations use only one port type.Schema1PortType

It is possible because all ports are one-way ports and use only one operation.

I have added a B orchestration. It helps to test the sample, showing all test messages in channel.

Orch.B

The Receive shape Filter is empty. A Receive Port (R_Shema1Direct) is a plain Direct Port. As you can see, a subscriptionB.Subscription expression of this direct port has only one part, the MessageType for our test schema: A Filer is empty but, as you know, a link from the Receive shape to the Port creates this MessageType expression.

I use only one Physical Receive File port to send a message to all processes.

Each orchestration outputs a Trace.WriteLine(“<Orchestration Name>”).

Forward Binding

This sample has three orchestrations: A_1, A_21 and A_22.

A_1 is a sender, A_21 and A_22 are receivers.

Here is a subscription of the A_1 orchestration:A_1.Subscription

It has two parts

  • A MessageType. The same was for the B orchestration.
  • A ReceivePortID. There was no such parameter for the B orchestration. It was created because I have bound the orchestration port with Physical Receive File port. This binding means the PortID parameter is added to the subscription.

All

How to set up the ports?

  1. All ports involved in the message exchange should be the same port type. It forces us to use the same operation and the same message type for the bound ports.
  2. This step as absolutely contra-intuitive. Ninja We have to choose a Partner Orchestration parameter for the sending orchestration, A_1. The first strange thing is it is not a partner orchestration we have to choose but an orchestration port. But the most strange thing is we have to choose exactly this orchestration and exactly this port.It is not a port from the partner, receive orchestrations, A_21 or A_22, but it is A_1 orchestration and S_SentFromA_1 port. Surprised smile
  3. Now we have to choose a Partner Orchestration parameter for the received orchestrations, A_21 and A_22. Nothing strange is here except a parameter name. We choose the port of the sender, A_1 orchestration and S_SentFromA_1 port.

As you can see the Partner Orchestration parameter for the sender and receiver orchestrations is the same.

Testing

I dropped a test file in a file folder. There we go: DebugView.Forward

  1. A dropped file was received by B and by A_1
  2. A_1 sent a message forward.
  3. A message was received by B, A_21, A_22

Let’s look at a context of a message sent by A_1 on the second step:

  • A MessageType part. It is quite expected.
  • A PartnerService, a ParnerPort, an Operation. All those parameters were set up in the Partner Orchestration parameter on both bound A-1.MessageSent.Contextports.

 

 

Now let’s see a subscription of the A_21 and A_22 orchestrations. A_2x.Subscription

Now it makes sense. That’s why we have chosen such a strange value for the Partner Orchestration parameter of the sending orchestration.

Inverse Binding

This sample has three orchestrations: A_11, A_12 and A_2.

A_11 and A_12 are senders, A_2 is receiver.

All

How to set up the ports?

  1. All ports involved in the message exchange should be the same port type. It forces us to use the same operation and the same message type for the bound ports.
  2. This step as absolutely contra-intuitive. Ninja We have to choose a Partner Orchestration parameter for a receiving orchestration, A_2. The first strange thing is it is not a partner orchestration we have to choose but an orchestration port. But the most strange thing is we have to choose exactly this orchestration and exactly this port.It is not a port from the partner, sent orchestrations, A_11 or A_12, but it is A_2 orchestration and R_SentToA_2 port. Surprised smile
  3. Now we have to choose a Partner Orchestration parameter for the sending orchestrations, A_11 and A_12. Nothing strange is here except a parameter name. We choose the port of the sender, A_2 orchestration and R_SentToA_2 port.

Testing

    I dropped a test file in a file folder. There we go: DebugView.Inverse

    1. A dropped file was received by B, A_11 and by A_12
    2. A_11 and A_12 sent two messages forward.
    3. The messages were received by B, A_2

Let’s see what was a context of a message sent by A_1 on the second step:

  • A MessageType part. It is quite expected.
  • A PartnerService, a ParnerPort, an Operation. All those parameters were set up in the Partner Orchestration parameter on both bound ports.A-1x.MessageSent.Context

Here is a subscription of the A_2 orchestration.

A-2.Subscription

Models

I had a hard time trying to explain the Partner Direct Ports in simple terms. I have finished with this model:

Forward Binding

Publisher/Sender doesn't know Subscribers/Receivers. Subscribers/Receivers know a Publisher/Sender.
1 Publisher –> M Subscribers

Inverse Binding

Publishers/Senders know a Subscriber/Receiver. Subscriber/Receiver doesn't know Publishers/Senders.
M Publishers –> 1 Subscriber

Notes

 

Orchestration chain

It’s worth to note, the Partner Direct Port Binding creates a chain opened from one side and closed from another.

The Forward Binding: A new Receiver can be added at run-time. The Sender can not be changed without design-time changes in Receivers.

The Inverse Binding: A new Sender can be added at run-time. The Receiver can not be changed without design-time changes in Senders.

BizTalk: Internals: Namespaces

BizTalk team did a good job hiding the .NET guts from developers. Developers are working with editors and hardly with .NET code. The Orchestration editor, the Mapper, the Schema editor, the Pipeline editor, all these editors hide what is going on with artifacts created and deployed.

Working with the BizTalk artifacts year after year brings us some knowledge which could help to understand more about the .NET guts.

I would like to highlight the .NET namespaces. What they are, how they influence our everyday tasks in the BizTalk application development.

What is it?

Most of the BizTalk artifacts are compiled into the NET classes. Not all of them… but I will show you later. Classes are placed inside the namespaces. I will not describe here why we need namespaces and what is it. I assume you all know about it more then me.

Here I would like to emphasize that almost each BizTalk artifact is implemented as a .NET class within a .NET namespace.

Where to see the namespaces in development?

The namespaces are inconsistently spread across the artifact parameters.

Let’s start with namespace placement in development. Then we go with namespaces in deployment and operations.

I am using pictures from the BizTalk Server 2013 Beta and the Visual Studio 2012 but there was no changes regarding the namespaces starting from the BizTalk 2006.

Default namespace

image

When a new BizTalk project is created, the default namespace is set up the same as a name of a project. This namespace would be used for all new BizTalk artifacts added to this project.

Orchestrations

When we select a green or a red markers (the Begin and End orchestration shapes) we will see the orchestration Properties window. We also can click anywhere on the space between Port Surfaces to see this window.

image

 image

Schemas

The only way to see the NET namespace for map is selecting the schema file name into the Solution Explorer.

image

image

Notes:

  • We can also see the Type Name parameter. It is a name of the correspondent .NET class.
  • We can also see the Fully Qualified Name parameter.
  • We cannot see the schema namespace when selecting any node on the schema editor surface. Only selecting a schema file name gives us a namespace parameter.
  • If we select a <Schema> node we can get the Target Namespace parameter of the schema.
    image
    image

    This is NOT the .NET namespace! It is an XML namespace. See this XML namespace inside the XML schema, it is shown as a special targetNamespace attributeimage

    Here this XML namespace appears inside the XML document itself. It is shown as a special xmlns attribute.
    image

     

Maps

It is similar to the schemas. The only way to see the NET namespace for map is selecting a map file name into the Solution Explorer.

image

image

Pipelines

It is similar to the schemas. The only way to see the NET namespace for pipeline is selecting a pipeline file name into the Solution Explorer.
imagez

image

Ports, Policies and Tracking Profiles

The Send and Receive Ports, the Policies and the BAM Tracking Profiles do not create the .NET classes and they do not have the associated .NET namespaces.

How to copy artifacts?

Since the new versions of the BizTalk Server are going to production I am spending more and more time redesigning and refactoring the BizTalk applications. It is good to know how the refactoring process copes with the .NET namespaces. Let see what is going on with the namespaces when we copy the artifacts from one project to another.

Here is an example: I am going to group the artifacts under the project folders. So, I have created a Group folder, have run the Add / Existing Item.. command and have chosen all artifacts in the project root. The artifact copies were created in the Group folder:

image

What was happened with the namespaces of the artifacts?

image

As you can see, the folder name, the “Group”, was added to the namespace. It is great! When I added a folder, I have added one more level in the name hierarchy and the namespace change just reflexes this hierarchy change.  The same namespace adjustment happens when we copy the BizTalk artifacts between the projects.

But there is an issue with the namespace of an orchestration. It was not changed. The namespaces of the schemas, maps, pipelines are changed but not the orchestration namespace. I have to change the orchestration namespace manually.

Now another example: I am creating a new Project folder and moving the artifacts there from the project root by drag and drop. We will mention the artifact namespaces are not changed.

Another example: I am copying the artifacts from the project root by (drag and drop) + Ctrl. We will mention the artifact namespaces are changed. It works exactly as it was with the Add / Existing Item.. command.

Conclusion:

  • The namespace parameter is put inconsistently in different places for different artifacts
  • Moving artifacts changes the namespaces of the schemas, maps, pipelines but not the orchestrations.

BizTalk: mapping with Xslt

BizTalk Map Editor (Mapper) is a good editor, especially in the last 2010 version of the BizTalk. But sometimes it still cannot do the tasks easily. It is time for the Xslt code, it is time to remember that the maps are executed by the Xslt engine. 

Right-click the Mapper Grid (a field between the source and target schemas) and choose Properties /Custom XSLT Path.  Input here a file name for the file with Xslt code. Only this code will be executed now, forget the picture in the Mapper, all those links and functoids. 

Let’s see the real-life example. It is the typical example of the "schema flattening". We have a target record and we have to create it from different unrelated source records.

As we can see, there are two source Addresses. One is on the top level and the second one is inside the Member_Address record with MaxOccurs=* .

A target address is placed inside the Locator record with MaxOccurs=*.

Map.Pict

The requirement is to map all source addresses to one target address.

The source Xml document looks like:

Source.Xml

The result Xml should be like this:

Output.Xml

Try to make up this mapping with the Mapper and you would spent good amount of time and the result map would be tricky.

If we use the Xslt code, the mapping will be simple and unambiguous, like this:

Xslt

There are two identical pieces creating the Locator records. 

The first one is started with <xsl: if test="s2:Address_1". It maps the upper level address to the Locator.

The second one is started with <xsl: for-each select="s2:Member_Address/s2:Member_Address". It also maps the addresses nested inside the /../Member_Address/Member_Address record to the Locator.

Simple, clear.

RabbitMQ Server: Error: Host name in Upper-case

This is a small thing, but very annoying.

I didn’t find it in Internet, so decided to blog it here.

My environment: Windows Server 2008 R2 x64; RabbitMQ 2.7.1.

I installed the RabbitMQ Server and tried to interact with it using the rabbitmqctl.bat script. This script requires to use the host name:

>rabbitmqctl –n rabbit@Win2K8R2 status

I was getting the error again and again:

“Error: unable to connect to node rabbit@Win2K8R2: nodedown”

RabbitServer.Status

 

I used the exact string provided me by script (see above in red) -

- nodes and their ports on Win2K8R2:…

and the error text was -

Status of node rabbit@Win2K8R2

When I changed the host name to upper-case, error was gone.

          >rabbitmqctl –n rabbit@WIN2K8R2 status

Conclusion:

   Use only upper-case letters in the host name in rabbitmqctl.bat script!

BizTalk and RabbitMQ

The same article in the TechNet.

If you are working with queues in BizTalk Server, the most possible it is the MSMQ (in my experience).

MSMQ is an old man of the Microsoft technology stack. It was created when there were no good standards for messaging. Now MSMQ is wrapped partly in the .NET System.Messaging namespace. It is just a small facelift. MSMQ is still a proprietary technology without well-defined messaging protocol. It means, you cannot use the MSMQ messaging protocol without MSMQ itself.

Now we see vortex of new messaging standards and technologies. One on the top is the AMQP standard and one of the bold implementation of AMQP is the RabbitMQ.

It is one more protocol, one more messaging system which, for sure, can be integrated with BizTalk Server.

Here I will implement the standard queue messaging: sending and receiving messages from BizTalk down through the RabbitMQ queue. I am using the WCF-Custom LOB Adapter (provided from BizTalk side) and the WCF RabbitMQ Binding (provided from RabbitMQ side) to link two systems.

Installation and preparation

Assuming the BizTalk Server and the Visual Studio 2010 are also installed.

Installing RabbitMQ Service

  1. Go to the RabbitMQ site and download the last versions of server and client for Windows. For me it was rabbitmq-server-2.7.1.exe and rabbitmq-dotnet-client-2.7.1-dotnet-3.0.zip. Client includes the WCF binding which I will use lately. There are several good manuals: rabbitmq-dotnet-client-2.7.1-user-guide.pdf and rabbitmq-dotnet-client-2.7.1-wcf-service-model.pdf. I was quite impressed by quality distributives and documentation.
  2. Start rabbitmq--server-x.x.x.exe. It will request to install Erlang. Agree, go to the Erlang site [http://www.erlang.org/] and download it. For me it was Erlang.otp_win32_R15B.exe
  3. Start Erlang…exe. Installation went smoothly.
  4. Start rabbitmq--server-x.x.x.exe again. Installation went smoothly.

RabbitMQ service is installed. You can see it in the Services window.

Creating RabbitMQ Client Base Assemblies

 Now it is time to install the client for .NET and WCF binding extension. Use this manual.

  1.  Downloaded the .msi. Installed it. Hmm… no Rabbit.Client.dll (When I started with Examples, they all need this file. It is a main client dll).

  2. Downloaded the *-dotnet-3.0.zip. Nope, it was wrong.
  3. We need the last rabbitmq-dotnet-client-2.7.1.zip file. Extract everything from this zip.
  4. Made a copy of the Local.props.example file to Local.props file in the same root folder.
  5. Start Visual Studio with RabbitMQDotNetClient.sln and successfully build it. The ..\rabbitmq-dotnet-client-2.7.1\projects\wcf\RabbitMQ.ServiceModel\obj\Debug\RabbitMQ.ServiceModel.dll was created. This dll together with the RabbitMQ.Client.dll are the assemblies used by the RabbitMQ clients.

Set up the RabbitMQ binding extension

Here I really lost a lot of time. It is easy for the C# clients; the RabbitMQ.ServiceModel.Test gives all the information. But we have to use this binding from the WCF-Custom LOB adapter. To do so we have to add the assemblies to GAC, change machine.config, etc. See the WCF LOP Adapter documentation.

  1. Sign the RabbitMQ.ServiceModel and RabbitMQ.Client projects and GAC-ed them.
  2. Insert  an <add> node into the config section of machine.config:

<extensions>
      <bindingExtensions>
        …
        <add  name="RabbitMQBinding" 
          type="RabbitMQ.ServiceModel.RabbitMQBindingSection, RabbitMQ.ServiceModel, Version=0.0.0.0, Culture=neutral,   PublicKeyToken=1751e286f1ab778d"/>
      </bindingExtensions>
    </extensions>

Note:

  • When the binding extension assembly is placed in GAC, we cannot use the above section in the app.config file! This <add> element should be placed only into the machine.config file. Otherwise it cannot be seen in the binding extension list of the WCF LOB Adapter.
  • Use >Gacutil.exe /l RabbitMQ.ServiceModel
    to get the real parameters for your assembly.
I have several machine.config files in folders:
    • C:\Windows\Microsoft.NET\Framework\v2.0.50727\CONFIG
    • C:\Windows\Microsoft.NET\Framework\v4.0.30319\Config
    • C:\Windows\Microsoft.NET\Framework64\v2.0.50727\CONFIG
    • C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Config

Which one to use? My VM is 64bit. BizTalk Host of the port is 32bit. BizTalk is the 2010 version. In my case the right file was the C:\Windows\Microsoft.NET\Framework\v4.0.30319\Config\machine.config.



Finally the RabbitMQ binding installed and we can see it in the Binding Type list for WCF-Custom adapter.

Testing

Creating the BizTalk test

I have created two pairs of port for testing.

File RP -> WCF-Custom (RabbitMQ) SP ->

 [RabbitMQ Server]

 -> WCF-Custom (RabbitMQ RP) -> File RP

 

I’ve changed only two parameters: “hostname” and “oneWay” and let others be default:

 

URL-s for the Send and Receive RabbitMQ Ports set up as “soap.amqp://192.168.11.128/Mytest” for both of them.

Let’s test it.

Small text file was dropped into In folder, consumed by Receive Port, went through the RabbitMQ queue, and appeared in Out folder. Success.

Note:

  • Now the RabbitMQ binding is set up to process only well-formed Xml documents.
  • “localhost” as “hostname” works fine.

Using the RabbitMQ Binding Element

The RabbitMQ binding is limited to the Text messaging encoding. If we need to change it or add more binding extensions, we have to use the customBinding type and use the rabbitMQBindingElement together with another element (including another encoding). How to do this?

1. Add the <add> node to the machine.config:

<bindingElementExtensions>
                           …
        <add name="rabbitMQBindingElement" type="RabbitMQ.ServiceModel.RabbitMQTransportElement, RabbitMQ.ServiceModel,   Version=0.0.0.0, Culture=neutral, PublicKeyToken=1751e286f1ab778d"/>
</bindingElementExtensions>

 2. Change the Receive Location transport parameters. It was the RabbitMQBinding. Change it to the customBinding. The Transport Binding Element has to be changed to the rabbitMQBindingElement. I changed only one parameter: hostname as “localhost”.

 

 3. Test it as previously.

Conclusion

Using RabbitMQ as one more transport for BizTalk and/or WCF applications is possible and straightforward. The learning curve is short. There are not too many issues with setting up the RabbitMQ Service and Client parts. Actually there wasn’t any RabbitMQ part which was exposed to the BizTalk directly. RabbitMQ works only with WCF-Custom adapter and only this adapter communicates with RabbitMQ Service.
Now the RabbitMQ implements limited client functionality for the WCF. RabbitMQ implements the AMQP v.0.9 protocol, not the last version 1.0.

Next steps

Next steps are to test this solution with different messaging patterns and try to figure out how to use RabbitMQ advantages (if there are some) over the MSMQ. Potentially the advantages are the simplest implementation of the Request/Response and Duplex patterns, better scalability, better performance, better manageability and support.

 

BizTalk and 0MQ (ZeroMQ)

It is stupid to compare those two beasts.

BizTalk Server is a pricey, huge development and run-time system, it includes tons of the tools, supports tons of protocols and systems. It works for very specialized middle ware integration stack. It integrates big and complex systems as SAP, CRM, Oracle and intermediates many protocols and standards as SOAP, EDI, SWIFT, FTP, tons of them.

It is a messaging system.

ZeroMQ is a low-level, wire-level protocol, the same level as TCP. It has ultra-simple API. It is implemented as a library for dozens languages as C#, Java, Python, PHP, C++, COBOL, Ada, Haskel, F#, Erlang, etc. It works in the most OS: Windows, Linux, QNX... It is free and works under open-source license.

It is a messaging system.

 

BizTalk implements many messaging exchange patterns: Request-Response, Pub-Sub, Queue, Message Broker, Router, Convoy.

0MQ implements the same: Request-Response, Pub-Sub, Pub-Sub with Topics, Queue, Message Broker, Router...

BizTalk development stack is compounded of several editors. BizTalk run-time stack includes several services on top of SQL Server, several tools, and tons of dll-s and SQL code.

0MQ is just a library, a dll. Library per language, per OS. No dedicated services.

...
Why I feel so comfortable with this 0MQ?
Why I think it is a serious choice to do messaging in my real-life messaging?
Do I want to abandon the comfort of a big track to ride almost bare-bone wheels?

I am not the only one who comparing 0MQ with big brokers. There is interesting article comparing RabittMQ (AMQP) and 0MQ.

BizTalk: Ordered Delivery

 

It is one more description of the Ordered Delivery (OD) in BizTalk.

The main article about it is in MSDN.

Here I am discussing the BizTalk Ordered Delivery “implementation details”.

OD Considerations

  • Ordered Delivery (sequential) mode is opposite of the “Parallel Delivery” mode. Parallel Delivery is the most productive mode; the Ordered Delivery is less productive mode.
  • Transports such MSMQ and protocols, supporting the WS-ReliableMessaging, are the protocols supporting OD. Other protocols as FTP, File, SQL, SMTP etc. do not have notion of the “order”.
  • BizTalk application usually is a part of the whole message workflow.
  • There are two approaches in the OD implementation:
    • All steps of the message workflow independently support OD.
    • A Destination System implements the re-sequencing and manages lost and duplicate messages.
  • Order is relative. Sequences can be ordered regards one or several parameters. For example, OD for the Company or for the Company + Department.

OD and the BizTalk Architecture

  • MessageBox is an implementation of the Message Queue. OD is an intrinsic feature of the MessageBox.
  • The BizTalk Server works in the Parallel Delivery mode by default.
  • There are three parts in the BizTalk message exchange outside of the MessageBox in relation to OD: Receive Locations; Orchestrations; Send ports.
    • Receive Locations support OD on the Transport level (say, MSMQ and some WCF adapters).
    • OD in Orchestrations is implemented by the sequential convoy pattern.
    • Send ports support OD for all static adapters.
  • The BizTalk Pipelines (part of Receive and Send Ports) always process messages in order using streams.

OD and Ports

To force Send Ports work in order we set up a flag the “Ordered Delivery” in Send Ports, then the BizTalk MessageBox takes care of implementing OD.

To force Receive Locations work in order we set up flag the “Ordered Delivery” option in the Receive Location Transports, whenever is possible. Then the BizTalk Transport Adapter takes care of implementing OD.

Ordered Delivery Send Port instance works as a singleton service. Since start it stays in Running state. It will not recycle if we restart its Host Instance. We could manually terminate it, if we want.

 

OD and Orchestrations

MessageBox implements the convoy message exchange pattern [See Using Correlations in Orchestrations]. See the detail convoy description in the BizTalk Server 2004 Convoy Deep Dive article.
It is not just a design pattern that developer should follow. There are special mechanisms inside MessageBox responsible for implementing OD.

OD and Orchestration: Sample

 

Imagine four Orchestrations which implement four approaches to the sequencing.



The first is the ProcessNoOrder Orchestration. It processes all messages without any order. One ProcessNoOrder Orchestration instance will be created for each inbound message.

The ProcessInOrder Orchestration processes all messages in one sequence. Only one ProcessInOrder Orchestration instance will be created.

The ProcessInOrderByCompany Orchestration processes messages in sequences correlated by the Company value (A, B, C, D, etc.). The separate queue is created for each new value of the Company. Messages inside queues are processed in order. Queues for different Companies are independent. A separate ProcessInOrderByCompany Orchestration instance will be created for each new Company value.

The ProcessInOrderByProduct Orchestration works exactly as the ProcessInOrderByCompany Orchestration but correlated by the Product value (xx, yy, etc.).

Discussions

Performance

By default all Orchestration and Messaging Service instances works in the Parallel Delivery mode with maximum performance.

If we check up the Ordered Delivery option in Send Port, BizTalk will initiate the Send Port instance as a singleton service. It is always a single instance. We don’t have the flexibility of the Orchestration where we could tune up “the proportion of the sequencing” and could control the recycling of the service instance.

Send Port OD could be in two states, on and off:

  • OD is off: a service instance per message, one message per queue, maximum performance.
  • OD is on: one infinite service instance, all messages in one queue, minimum performance.

Orchestration OD could be in two states also, on and off:

  • OD is off: a service instance per one activating message, one activating message per queue, maximum performance.
  • OD is on: one or several service instances, one per new correlation set value; all correlated messages per queue; performance is from min to max, depends ot the correlation set..

Carefully designing the correlation sets we could tune up the performance of the Orchestration. For example, if we only care of the document order for each separate Company, we include the Company to the Correlation set. If we had thousand documents related to hundreds companies, the performance will be near maximum. If there are only two companies, the performance will be near minimum, and we should consider improving the correlation with one more parameter.

Orchestrations and Zombies

Zombies are big problem of Convoy Orchestrations. See BizTalk: Instance Subscription and Convoys: Details article with description of this problem. This problem could be mitigated but could not be completely removed. We are waiting a new version of the BizTalk Server where this issue will be targeted.

BizTalk Server version Next, Ordered Delivery and Zombies

It is possible the BizTalk Server version Next will implement the automatic Ordered Delivery for Orchestrations, with pattern similar to the Ordered Delivery in Send Ports.

Three new Orchestration parameters are shown up there: Ordered Delivery, Stop on Exception, and Recycle Interval (in seconds).

 



Ordered Delivery parameter works in the same way as the Ordered Delivery parameter of the Send Port. Now we don’t have to build the Convoy Orchestration manually. No more Loop inside Orchestration.

If the Ordered Delivery parameter set up to True, the Orchestration is working as a Singleton. The first Receive shape receives all correlated messages in sequence. Correlation set is created implicitly regards of the Activation Subscription expression.

There are several limitations for this kind of Orchestration. The most important is: only one Receive shape is permitted here.

There are two big advantages of this new feature:

  • Simplified Orchestration design for the Ordered Delivery.
  • No more Zombies. The Orchestration instance is recycled in controllable way, when no messages, matched the Orchestration Subscription, are placed in the MessageBox.

Conclusion

We discussed the Ordered Delivery implemented in the BizTalk Server and ways to improve it.

 

BizTalk: Advanced Questions

This article is a Part 6 of the Interview Questions series.

It is time for fun!

These questions are for an Advanced level of the BizTalk Developers. I hope you don’t have a single chance to answer these questions, if you don’t have a real life experience in BizTalk Server projects.

If you have more advanced questions, please, comment them down and I will add them to this list under your name.

I will constantly add more questions into the end of list. Be in tune.

Advanced level

  • The error in BizTalk usually creates a suspended message. But routing error creates two messages. Why so? How we could utilize this feature?
  • Send ports have an “Ordered delivery” option. How it works for a File adapter, if the File protocol does not have a notion of file order?
  • Send ports have an “Ordered delivery” option but Receive ports do not have it. Why so?
  • You set up an “Ordered delivery” option in a Send Port. What is special about a Service Instance of this Port?
  • How to force an Orchestration do the ordered delivery? Is there a special “Ordered delivery” option in an Orchestration? If it is not, why?
  • Send ports have the “Priority” options, but Receive ports and Receive Locations do not have them. Why so?
  • How does the Priority feature of the Send Port work?
  • Why the Priority option can be set up only for the Send Port?
  • Send ports have the “Retry” options, but Receive ports and Receive Locations do not have them. Why so?
  • How does the Retry feature of the Send Port work? In what scenarios this feature is helpful, in
  • Send ports have a “Backup Transport” but Receive ports do not have it. Why so?
  • Send ports have a “Filter” but Receive ports do not have it. Why so?
  • Send ports have the “Maps” and “Transport”; Receive ports have “Maps” but “Transport” is associated with Receive Locations. Why so?
  • What is the difference between “Send Port Group – Send Port” relation and “Receive Port – Receive Location” relation?
  • Receive Locations have the Schedule parameters for date and time but Send Ports have the Schedule parameters only time. Why so?
  • Can we create two Send Ports with the same URL? Can we create two Receive Locations with the same URL? Why so?
  • Filter expression for the Send Port is defined at run-time, but Filter expression for the Orchestrations is defined only at design-time. Why so?
  • We created a custom pipeline component. Do we have to place it into the “<BizTalkFolder>\Pipeline Component” or install it to GAC or both?
  • How does an Orchestration make sure the message was sent successfully by one-way Send port? How does it changed for a two-way Send port?
  • What is it the “CodeList Database” property of the Schema? How to use it?
  • What is the “Property Schema Base” property of the Property schema element? It has the values: MessageDataProperyBase, MessageContextPropertyBase, and PartContextPropertyBase. What is the difference between using MessageDataProperyBase and MessageContextPropertyBase?
  • We bind an Orchestration with a Send Port. What does it do regards of the subscription?
  • We bind an Orchestration with a Receive Port. What does it do regards of the subscription?
  • How do Filters of the Send Ports and Send Port Groups work together? Say, a Send Port belongs to a Send Port Group. If this Send Port and this Send Port Group have the same Filter expression, does it create two subscriptions and, as a result, two messages are sent to the Send Port?
  • Why do we need a special Property schema for Promoted properties? What is the purpose of this schema? Distinguished properties do not use this kind of schema. Why so?
  • Properties of some BizTalk artifacts have a Tracking option. Right-click menu of some BizTalk artifacts have the “Tracking…” command. Why there is such a difference between artifacts? Why Maps do not have any Tracking options? Why Schemas have only Tracking property? Why Send Port Groups do not have any Tracking options? Why Receive Locations do not have any Tracking options? Why BizTalk Applications do not have any Tracking options?
  • When a BizTalk Assembly is deployed, is it stored in the BizTalk database or is it just referenced in the BizTalk database?
  • Why a BizTalk Assembly must be installed into GAC? Why deploying it to BizTalk database is not enough? 
  • Can you describe at least three scenarios where smart Orchestration Engine finds the errors in the messages patterns at design time (while build a BizTalk project)?
  • A Host has an "Authentication Trusted" option. What does it mean? What implication it has to the message processing? How do we have to use it / don't use?
  • Could we create an XLANGMessage in the C#/VB code? Please, provide code.
  • We can use an XmlDocument class as a Message type for receiving the untyped Xml messages in Orchestration. What .NET classes do we have to use for receiving the untyped text or binary messages in Orchestration. Why so?
  • Is the Publish-Subscribe BizTalk Engine API well defined? Where? If not, why so?
  • Does the BizTalk use the strategy “First  received, first processed” or “First received, last processed” of “First received, first processed, first sent” of something else? Why so?
    Describe scenarios, when this strategy works better, and scenarios, when this strategy works badly.
  • Are Orchestration Instances processed in order? Why so? Describe scenarios.
  • Is it possible to force the Hydration / Dehydration of the Orchestration instances, Messaging instances? How to do this? Why do this?
  • In what scenarios  will  we use the Compensations in Orchestrations? Why so?
  • The Atomic scopes in Orchestrations have a Retry feature. How does it work? Describe scenarios when it is used.
  • What serializer does the BizTalk use, an XmlSerializer or a DataContractSerializer? What implication has it for the real life scenarios?
  • What transform engine is the BizTalk using, an XslTransform or an XslCompiledTransform? What implication  has it for the real life scenarios?
  • The BizTalk Business Rule Engine uses a Rete engine. In what scenarios this Rete engine is the more efficient / less efficient?
  • Let’s classify the applications as Front-end, Middle-level and Back-end. Where are the best places to use BizTalk? Please, provide scenarios. What are the most important features we have to take in account?
  • What BizTalk features are the differentiators, when we compare the BizTalk Server with other alternatives? What are those alternatives?
  • What is a Heartbeat in the BizTalk? Why is it implemented? Can we change the heartbeat interval? How does this interval influence the processes in the MessageBox? 
  • What is the difference, if any, between the ESB and the Broker conceptions?
  • How to set up a custom name for a dynamic send port?
  • What kind of the dynamic ports are existed? Are there dynamic send  ports? Are there dynamic receive ports? Why is it so?

BizTalk: Samples: Error Handling, Email Notification

One problem with email notifications is it usually sends too many emails. Errors frequently happen in series. For example, a temporarily issue with network can result in hundreds repetitive emails. In result an email notification designed as a “VIP” notification is downgraded to the “annoying” status.

This email flooding could be mitigated with this solution.

A Sample demonstrates using the emails for sending notification in case of failed messages.

It compounds several similar errors in one email notification.

 

Recently I've uploaded this sample 

BizTalk: Samples: ErrorHandling: Notification emails

to MSDN Gallery.

It has all solution artifacts and can be used in real-life projects.

 

BizTalk: Sample: Error Handling

Recently I uploaded a "BizTalk: Sample: Error Handling" in MSDN Gallery

This sample shows the BizTalk features:

  • Routing Failed Messages in ports
  • Catching Failed Messages with Send Port
  • Catching Failed Messages with Orchestration
  • Handling Exceptions inside Orchestration
  • Handling SOAP Fault messages inside Orchestration

BizTalk Orchestration Naming Conventions

 This is a second part of the BizTalk Naming Convention.

Special Orchestration Objects 

<Message> =: 
      msg_ + <ShortMessageType>

<Variable> =: 
      var_ + <Name>

<CorrelationSet> =: 
      cor_ + <Name>

<OrchestrationParameter> =: 
      par_ + < Name>

<RoleLink> =: 
      roleLink_ + <Name>

Note: These objects are special BizTalk objects. They are used in different language context and sometime they use different language syntax. Prefixes help to differentiate the objects in the XLang expressions.

<Port> =: 
      <prefix> + <Name>

where

Send

Receive

Send-Receive
(Solicit-Response)

Receive-Send
(Request- Response)

prefix

S_+

R_+

SR_+

RS_+

Notes:

  • The Port objects are the only objects which can be seen outside orchestration. We see them while bind orchestration with Ports, not the orchestration Ports but Ports created in the BizTalk Administration Console. Sometimes the orchestration Ports are named as Logical Ports and the Ports as the Physical Ports. Here are some considerations about this ambiguity with Port names.
  • Send-receive prefixes help while Orchestration is binding with Ports. Ports with different Communication pattern are using different prefixes.

Example: S_ OrderAck.

 

Orchestration Object Types

<ArtifactType> =:
      <ArtifactName> + “_type

Note: We can see orchestration types in the Orchestration View window in Types. They are: Port, Multi-part Message, Correlation and Role Link Types. We can use one suffix the “_type” for all different types because different types are seen only in the different lists and never mixed. For instance, we can never see the port types together with message types.

Controversial:Suffixes for types work better than prefixes, because types are mostly used in the drop-down lists not in the XLang expressions.

Orchestration Workflow Shapes

Problems with orchestration shapes:

  •         Shapes are too small to display long names (only 12-18 characters are visible).
  •         We have to hover a mouse over a shape or click shape to see Properties window and to "understand" this shape, to understand which message is processed.

Useful features:

  •          Shapes have names, but names are not the “variable names”, not identifiers. They are descriptions (excluding the Port shapes names, which have not Name parameter but Identifier and Description parameters). Shape names are used only for description not as XLang variable identifiers. Shape names can be long and can include any symbols as spaces, dots, etc.
  •          Icons of shapes give us the useful information. Do not repeat the “icon information” by words. For example, we can a name a Construction shape as “Construct OrderAck” or as “OrderAck”. Last variant give us more clear and short definition because we have the Construction icon + name.
  •          Shape names are used mainly in Orchestration Editor (excluding the Port shapes names). We don’t need the “well-sorted” names, so we don’t need to use prefixes, because the main purpose of the prefixes is creating well-sorted lists.
  •          Group shape is a scalable shape. Nesting other shapes to a Group shape can make a long description visible. Group shape will display as much text as you want. Group shapes add a lot of documentation value to the orchestration.

Rules for shapes

  •          Whenever it is possible use the short MessageType as a shape name.
  •          Do not repeat a type of shape icon by word.
  •          Feel free to use spaces and any other symbols inside the shape names.
  •          Feel free to repeat the shape names.

Note: Purpose of the orchestration and the most of the shapes is in processing the messages. We can unambiguously describe the messages by the message type. A message type gives us the main information about a message. That is why in the most cases using the message type as the shape name gives us the main information about message flow, about the whole orchestration processing.
Example: Send shape with name "OrderAck" means “send OrderAck message”.

Controversial:

  • Orchestration Naming Convetion works not to make easier the administration and troubleshooting but mostly to make orchestration more readable. Most orchestration names are not visible outside the Orchestration Editor. So, do not force the orchestration naming convention to hard! If your developers use differen conventions, it is fine; of course, if they understand each other. Forcing convention can do more harm than help. It can force team to spend more effort for a little.
  • When exception is thrown from a shape, the error description includes a name of the shape. It is a rare case when names are exposed outside of the Orchestration Editor. When we use MessageType as a name of shapes, many shapes can get the same name. So, if we want to differentiate shape names for debugging purpose, we could use numbers or single letters in the end of names.
    Example: “OrderAck 2”

 

BizTalk: Sample: PGP Encryption/Decryption Pipeline Components

A sample demonstrates the PGP Encryption/Decryption in pipelines.

You can download code here.
		
A sample is based on a sample by Brian Jones See original code here
		

The main additions to original code:

·         Single pipeline component was separated to two Encrypt and Decrypt pipeline components. It simplifies the pipeline configurations.

·         Configuration parameters are stored in SSO, which, I hope, improves security.

·         File names for temporary files are regenerated randomly each time. That eliminates errors in case when temporary file names are based on the inbound file names and pipelines are working simultaneously in several ports.

To build pipeline components you have to download a BouncyCastle.Crypto.dll assembly from http://www.bouncycastle.org/csharp .

The solution includes two pairs of the PGP keys. You can generate yours pairs using, for example, PortablePGP utility http://primianotucci.com

Configuration includes a config file for a SSO Config Store utility created by Richard Seroter.

The test pipeline project includes four pipelines:

·         Send and Receive pipelines for Encryption

·         Send and Receive pipelines for Decryption.

To test these pipelines I’ve created four receive ports and four send ports. They create four test workflows:

·         Encryption on a Receive port:
RP::GLD.Samples.Pipelines.Encrypt.Encode  ( PgpEncryptReceive pipeline )
 
è SP:: GLD.Samples.Pipelines.Encrypt.PassThru

·         Encryption on a Send port:
RP::GLD.Samples.Pipelines.Encrypt.PassThru  
 
è SP:: GLD.Samples.Pipelines.Encrypt.Encode ( PgpEncryptSend pipeline )

·         Decryption on a Receive port:
RP:: GLD.Samples.Pipelines.Decrypt.Decode ( PgpDecryptReceive pipeline )
 
è SP:: GLD.Samples.Pipelines.Decrypt.PassThru

·         Decryption on a Send port:
RP:: GLD.Samples.Pipelines.Decrypt.PassThru
 
è SP:: GLD.Samples.Pipelines.Decrypt.Decrypt ( PgpDecryptSend pipeline )

To test pipelines:

·         Use test text files in a Tests\TestData folder

·         To test encryption:

1.        Copy test files to a Tests\Encrypt\In folder

2.        Encrypted files are created in a Test\Encrypt\Out folder.

·         To test decryption:

3.        Copy test encrypted files from a Test\Encrypt\Out folder to a Tests\Decrypt\In folder

4.        Decrypted files are created in a Test\Decrypt\Out folder.

 

BizTalk and WCF adapters: Operation and SOAP Action

One area here is not so well documented.

When we are filling in the Transport properties for WCF adapter, we see the SOAP Action header section.

What is it? Where we could take the values for Operations and Actions?

If we use the Consume WCF Service wizard, this section is filled up automatically. But sometimes we have to fill it in manually.

NP

Open a WSDL for the Web-service and search for a binding section. If you add the “?wsdl” to the Web-service URL, usually the Web-service WSDL is opened.

That’s how operations and actions in the WCF parameters and WSDL are mapping:

Hope it helps.

WCF: Questions for studying and interview: Routing Service

This article is a Part 5 of the Interview Questions series.

WCF, Routing Service

Q&A

• What are the features of the Routing Service (RS): Protocol bridging; Message transformation; Content-based routing; Application pools; Dynamic configuration; SOAP processing; Service hosting; Error handling; Backup endpoints?
• Which routing scenarios can RS implement: Service aggregation; Interface recomposition; Service versioning; Service chain; Priority routing?
• Is it possible to modify the routing configuration at run time?
• What is the RS doing if there is no match for a message routing?
• Can we route to the mix of the one-way and two-way destination endpoints using a single service endpoint?
• Do we have to create the implementation of the RS as a .NET class or an .svc file with “Service="System.ServiceModel.Routing.RoutingService, …” plus a configuration file is enough?
• What is it a SOAP processing? How it works?
• Can it convert messages between Soap 1.1/Soap 1.2/Soap 2.0 and/or WS-Addressing August 2004/WS-Addressing 1.0?
• What is it a Backup endpoint?
• What is the maximum number of the Backup endpoints?
• What is it a Protocol bridging?
• What is it a Content-based routing?
• How the Priority routing is working?
• How we can host the RS?
• Are there limits on using streaming for the RS? Where we have to buffer the messages?
• How we can expose mex endpoints of the original services through the RS?
• Can we use the RS as a load balancer?
 

WCF: Questions for studying and interview: Discovery

Q&A: WCF, Discovery
·         When the WS-Discovery standard gets approval first time?
·         Is the WS-Discovery using the XML format for messages? 
·         Is the WS-Discovery using the SOAP format for messages? 
·         What are the versions of the WS_Discovery protocol?
·         What version is the default version?
·         What is a difference between the Add-hoc and Managed modes? What are the Pro and Cons?
·         Describe the message exchange sequences of the Discovery protocol.
·         What are the message exchange sequences in the Add-hoc and Managed modes?
·         Can the Add-hoc and Managed modes be switched dynamically?
·         What messages are multicast and what are unicast?
·         What is the Discovery Proxy service?
·         It there a default Discovery Proxy service?
·         What is the difference between the Probe and Resolve requests?
·         What are the response messages for the Probe and Resolve requests?
·         What are the response messages for the Probe and Resolve requests if there is no match?
·         Who sends the announcement message, service or client?
·         What announcement messages are used?
·         Is announcement using a Special endpoint?
·         Is announcement optional or mandatory?
·         How to make a Web-hosted service discoverable? What two options do we have? How AppFabric or Discovery Proxy service is used to help with it?
·         What is a FindCriteria class? What are the Search criteria?
·         How long a client is waiting the results from the discovery request?
·         What happens if a discovery request didn’t get you any service endpoint?
·         Can we use different versions together on a single service host?
·         What endpoints are used for the Discovery?
·         What is a difference between the DiscoveryEndpoint and the DynamicEndpoint? Do we have to use a DynamicEndpoint with a DiscoveryClient?
·         What Discovery endpoints are the system endpoints?
·         How to make a custom endpoint discoverable?
·         Why the UPD protocol is used for discovery? What pros and cons?
·         How to use the Discovery Client Channel? What are alternatives?
·         What are the pros and cons using the Discovery Client Channel?
·         How to create a DiscoveryEndpoint on demand?
·         Can you compare the WS-Discovery and the UDDI?

 

Microsoft 2011 MVP Global Summit: Pictures

Photos from Summit

Mighty Canada

Mighty Canada

Oh, Canada!

Oh, Canada!

Microsoft is hearing you, guys.

Microsoft is hearing you, guys.

New Zealand is here.

New Zealand is here.

Portugal and Brazil are here...

Portugal and Brazil are here

France is here...

France is here...

Netherlands is here...

Netherlands is here...

all BizTalkiens

all BizTalkiens

and the whole world is here...

[See more MVP-s]

Vsem privet!

 

BizTalk: Using context for routing

Imagine the project where most of the routing happens between orchestrations. I.e. routing is mostly between the MessageBox and orchestration with direct endpoints.
Imagine also the most of the messages are with the same Message type.
Usually in this case messages got the special node only for the routing. For example, the field can be the “Originator” or “Recipient” or “From” or “To”.
What wrong is with this approach, it creates the dependency between the message and the message processing. Message “knows” something about Originator or Recipient.
So what we can do with it? How can we “colorize” the same message to route it to the different places without changing the message itself?
One of the decisions is to use the message context.
BizTalk uses the promoted properties for routing.  There are two kinds of the properties: the content properties and the context properties. The content property extracts its value from inside the message, it is a value of the element or attribute. [See MSDN] The context property gets its value from the message environment. It can be the port name that receive this message, it can be the message Id, created by the BizTalk. Context properties look like the headers in the SOAP message. Actually they are not the headers but behave like headers. The context properties are the good match for our case.
First, we don’t have to change the message itself to set or change the routing property. The context is stored outside the message body.
Second, we don’t have to create the property schema to use the context properties. [See MSDN: How to create Property schema] BizTalk has the predefined schema set for the context properties. [See MSDN: Message Context Properties] Use one of them and that's it.
The main purpose of the context properties is working on behalf of the BizTalk internals. But we can read, create and change them. Just do not interfere with BizTalk internals on this way.

 

BizTalk: Sample: Context routing and Throttling with orchestration

The sample demonstrates using orchestration for throttling and using context routing.
Usually throttling is implemented on the host level (in BizTalk 2010 we can also use the host instance level throttling). This sample demonstrates the throttling with orchestration convoy, that slows down a message flow from some customers. Sample implements sort of quality service agreement layer for different kind of customers.
The sample demonstrates the context routing between orchestrations. It has several advantages over the content routing. For example, we don’t have to create the property schema and promote properties on the schemas; we don’t have to change the message content to change routing.
Use case:
 The BizTalk application has a main processing orchestration that process all input messages. The application usually works as an OLTP application. Input messages came in random order without peaks, typical scenario for the on-line users.
But sometimes the big data batch payloads come. These batches overload processing orchestrations. All processes, activated by on-line users after the payload, come to the same queue and are processed only after the payload. Result is on-line users can see significant delay in processing. It can be minutes or hours, depending of the batch size.
Requirements:
On-line user’s processing should work without delays. Big batches cannot disturb on-line users. There should be higher priority for the on-line users and the lower priority for the batches.
Design:
Decision is to divide the message flow in two branches, one for on-line users and second for batches. Branch with batches provides messages to the processing line with low priority, and the on-line user’s branch – with high priority.
All messages are provided by one hi-speed receive port.
BTS.ReceivePortName context property is used for routing. The Router orchestration separates messages sent from on-line users and from the batch messages.
But the Router does not use the BizTalk provided value of this property, the Router set up this value by itself. Router uses the content of the messages to decide if it is from on-line users or from batches.
The message context property the BTS.ReceivePortName is changed respectively, its value works as a recipient address, as the “To” address for the next recipient orchestrations. Ideally we should use two receive ports or two receive locations and BizTalk set up the context property for us. If we use the same context for both messages (the same port and location), we need to use orchestration to modify the context property using content of the messages or something else.
Those next orchestrations are the BatchBottleneck and the MainProcess orchestrations.
Messages with context equal “ToBatch” are filtered up by the BatchBottleneck orchestration. It is a unified convoy orchestration and it throttles the message flow, delaying the message delivery to the MainProcess orchestration.
The BatchBottleneck orchestration changes the message context to the “ToProcess” and sends messages one after another with small delay in between. Delay can be configured in the BizTalk config file as:

                <appSettings>
                                <add key="GLD_Tests_TwoWayRouting_BatchBottleneck_DelayMillisec" value="100"/>
                </appSettings>
 
Of course, messages with context equal “ToProcess” are filtered up by the MainProcess orchestration.
 

NOTES:
  • Filters with string values: In Orchestrations (the first Receive shape in orchestration) use string values WITH quotes; in Send Ports use string values WITHOUT quotes.
  • Filters on the Send Ports are dynamic; we can change them in run-time. Filters on the Orchestrations are static; we can change them only in design-time.
  • To check the existence of the promoted property inside orchestration use the Expression shape with construction like this:
          if (BTS.ReceivePortName exists myMessage) { …; }
    It is not possible in the Message Assignment shape because using the “if” statement inside Message Assignment is prohibited.
  • Several predefined context properties can behave in specific way. Say MessageTracking.OriginatingMessage or XMLNORM.DocumentSpecName, they are required some internal rules should be applied to the format or usage of this properties. MessageTracking.* parameters require you have to use tracking and you can get unexpected run-time errors in some cases. My recommendation is - use very limited set of the predefined context properties.
  • To “attach” the new promoted property to the message, we have to use correlation. The correlation type should include this property. [Here is a good explanation by Saravana ]
  • Say we have two correlation types, each with one parameter inside. If we want to correlate to both parameters simultaneously, we cannot use both correlation types, we have to create the correlation type with both parameters and use this type. Otherwise only one correlation type would be chosen for correlation, that means only parameter would be used for correlation.