- Orchestrations

BizTalk: xpath: How to work with empty and Null elements in Orchestration

Leonid Ganeline — Tue, 17 Nov 2009 23:55:12 GMT

The problem is with three Empty-Null cases.

Is it possible to separate all these cases in Expression shapes of the Orchestration?

For example, we have the record with <name> element, in such flawors:

case: "NonEmpty"

<ns0:People>
<ns0:Name>Name_0</ns0:Name>
<ns0:IsDependent>IsDependent_0</ns0:IsDependent>
..

case: "Empty"

  <ns0:People>
    <ns0:Name></ns0:Name>
    <ns0:IsDependent>IsDependent_0</ns0:IsDependent>
..

case: "OneTag"

<ns0:People>
<ns0:Name/>
<ns0:IsDependent>IsDependent_0</ns0:IsDependent>
..

case: "Null"

<ns0:People>

<ns0:IsDependent>IsDependent_0</ns0:IsDependent>
..

Is it possible to separate all these cases in Expression shapes of the Orchestration?

There is no information into the MSDN about this [http://msdn.microsoft.com/en-us/library/aa561906(BTS.10).aspx]

I tried to use the xpath() function in two variants, one with "string(xpath_expression)" second with "xpath_expression"

Expression Shape:

[

System.Diagnostics.Trace.WriteLine("== with string() ====================================================================");

var_xpathString = "string(/*[local-name()='Root' and namespace-uri()='http://MapTest.IncPerson']/*[local-name()='People' and namespace-uri()='http://MapTest.IncPerson']/*[local-name()='Name' and namespace-uri()='http://MapTest.IncPerson'])";

System.Diagnostics.Trace.WriteLine("[" + System.Convert.ToString(xpath (msg_SourceRoot, var_xpathString)) + "]");

if ( xpath (msg_SourceRoot, var_xpathString) == null)
{ System.Diagnostics.Trace.WriteLine("Name == null"); }

else if ( xpath (msg_SourceRoot, var_xpathString) == "")
{ System.Diagnostics.Trace.WriteLine("Name == Empty");}

else
{ System.Diagnostics.Trace.WriteLine("Name != null && Name != Empty"); }

System.Diagnostics.Trace.WriteLine("-- no string() --------------------------------------------------------------------");
var_xpathString = "/*[local-name()='Root' and namespace-uri()='http://MapTest.IncPerson']/*[local-name()='People' and namespace-uri()='http://MapTest.IncPerson']/*[local-name()='Name' and namespace-uri()='http://MapTest.IncPerson']";

System.Diagnostics.Trace.WriteLine("[" + System.Convert.ToString(xpath (msg_SourceRoot, var_xpathString)) + "]");

if ( xpath (msg_SourceRoot, var_xpathString) == null)
{ System.Diagnostics.Trace.WriteLine("Name == null"); }

else if ( xpath (msg_SourceRoot, var_xpathString) == "")
{ System.Diagnostics.Trace.WriteLine("Name == Empty"); }

else
{ System.Diagnostics.Trace.WriteLine("Name != null && Name != Empty");}

]

I.e. the "string(xpath_expression)" expression is used in the firs section, the"xpath_expression" is used in the second.

Result is:

"NonEmpty" == with string() ====================================================================
[Name_0]
Name != null && Name != Empty
-- no string() --------------------------------------------------------------------
[Microsoft.XLANGs.Core.Part+ArrayBasedXmlNodeList]
Name != null && Name != Empty

"Empty"== with string() ====================================================================
[]
Name == Empty
-- no string() --------------------------------------------------------------------
[Microsoft.XLANGs.Core.Part+ArrayBasedXmlNodeList]
Name != null && Name != Empty

"OneTag"== with string() ====================================================================
[]
Name == Empty
-- no string() --------------------------------------------------------------------
[Microsoft.XLANGs.Core.Part+ArrayBasedXmlNodeList]
Name != null && Name != Empty

"Null"== with string() ====================================================================
[]
Name == Empty
-- no string() --------------------------------------------------------------------
[]
Name == null

Conclusion:

* I cannot separate the cases "Empty" and "OneTag"

* I can separate the cases "Empty and "Null" with "xpath_expression", not with "string(xpath_expression)" expression
* "Null" case does not throw an exception.
* xpath expression inside string() works fine to get all three cases ("Empty", "OneTag", "Null") under one "if" statement. And here it returns only "" (Empty string).

BizTalk: Naming convention for the BizTalk solutions

Leonid Ganeline — Fri, 31 Jul 2009 17:44:57 GMT

Naming convention for the BizTalk solutions

Version 2.1, 2009-08-09

1 How to use this document

To accommodate this document for the specific solution:

· Create the real names in Appendix or create a separate Dictionary document. See Instructions in Appendix.

· Review the text, marked with [TBD] flag, and change it for your case.

2 General rules

The “Naming guidelines for the .NET Framework types” [ http://msdn.microsoft.com/en-us/library/xzf533w0(VS.71).aspx ] are used as a basis of this document. Also see the “Naming Convention” in Wikipedia [ http://en.wikipedia.org/wiki/Identifier_naming_convention ]

Names should be short, sortable, readable, discoverable, and self-described.

The main test for constructing name rule should be in questions:

· In what kind of places can I see this name?

· How easily can I work with the name in these places?

“Work” means to read, to find, and to understand sense without errors. Usually we work with names in lists. Think about these places like about namespaces in programming languages.

Short names:

Create as short as possible name.
Use abbreviations only in restricted cases. See article the “Abbreviations”http://msdn.microsoft.com/en-us/library/141e06ef(VS.71).aspx
Use prefixes and suffixes only to differentiate names.
If you see one word in several places of the full name, consider this as a bad signal. Try to redesign the terms used in the names.

Sortable:

Create “sortable” name. That means using more generic/important part of the name in the leftmost position. For example, prefer the name Folder_20090515 than the name Folder_05_15_2009.

Readable:

See http://en.wikipedia.org/wiki/Typography

Use the name case compatible with the well-known practices in programming languages/protocols with respect to upper and lower cases. For instance, the XML namespaces (URL) should be in lower case format, but the other names should be in the Pascal format. See the “Capitalization Styles” article [http://msdn.microsoft.com/en-us/library/x2dbyw72(VS.71).aspx ]
If the word with specific case is widely used in company, don’t force to change it to the Pascal format.
Decorate infixes, prefixes or suffixes with lower case and with undrscore. For example: TicketBatch_type, Source1 _and_Source2_to_Target, msg_MyRequest.

Discoverable:

The name should be discoverable. That means we easily should understand by the name of the artifact where we can find the artifacts or additional information about it. Names should link artifacts. Say, the schemas with XML namespace the http://domain.company.com/solution/project/2009-05-14 definitely should be in the project the Company.Domain.Solution.Project and in assembly the Company.Domain.Solution.Project.dll.

Self-described (Semantics):

Create name form the “business point of view” not form the “developer point of view”, especially the name exposed outside solution.
Don’t use generic terms in the names. Examples: Send, Receive, Service, Message, Transformation, Schema, Map, Orchestration, BizTalk.
Place a frequently used term into the shared dictionary with comment about where do use it and do not use.

Full, Short, and Composited names:

Several artifacts have the full names (composited names) and short names. Usually we can easily understand what name we use in the specific context.

· If we need to add one or more logical grouping into the name, use the composited names created with words separated with dot. In specific contexts the separation symbol can be different, like underscore, dash or backslash. Each <…ShortName> can be a composite name. Composite names can be used for each part of the full names, as Company, Domain, Solution, and Project. For example, you can use name the Schemas.Niem for the project short name.

3 General names

3.1 Syntax

3.2 Examples

· Solution name: short name – MySolution, full name - MyCompany.MyDomain.MySolution

· Project name: short composite name – Schemas.Niem; full name - MyCompany.MyDomain.MySolution.Schemas.Niem

4 Names inside BizTalk Solutions

4.1 Solutions, BizTalk applications

We use term the solution in the Visual Studio meaning, it is a name of the solution we see in the Solution Explorer window in the Visual Studio. Sometimes we mix solution and project terms but not in this document.

BizTalk application is named as solution.

For example, for the MySolution solution use the MyCompany.MyDomain.MySolution full name.

4.2 Projects, Assemblies, .NET namespaces

We use term the project in the Visual Studio meaning, it is a name of the project we see in the Solution Explorer window in the Visual Studio.

For example, for the MyProject project use the MyCompany.MyDomain.MySolution.MyProject project full name, the MyCompany.MyDomain.MySolution.MyProject assemblyname, and the MyCompany.MyDomain.MySolution.MyProject namespace.

We should separate the .NET namespaces and the XML namespaces. They are different things and used in different places. For brevity we use term the namespace exactly for the .NET namespace and always use the term the XML namespaces for the XML namespaces.

Note: After creating a new project, copy the project name property to the project properties the “Assembly Name” and the “Default Namespace”.

4.3 Folders

Use the predefined folder structure on all development machines and, if it is possible, on the Test and Product servers. This eliminates the errors and deployment effort because several BizTalk artifacts use the absolute file paths in configuration.

<SolutionsRootFolder> =: c:\Solutions

For example, the MyCompany.MyDomain.MySolution.MyProject project is placed in the c:\Solutions\MyCompany.MyDomain.MySolution\MyProject folder.

5 XML namespaces

Use very strict rules for the XML namespaces, because the XML documents expose the data interfaces to the outer world. The XML namespaces should follow the industry standards and the corporate standards. The exposed interfaces are immutable, why we should use versioning for XML namespaces. XML namespaces work as global unique identifiers for nodes of the XML documents.

We must use the URL or URN formats to XML namespaces. Feel free to use one of these standards. See the “Namespaces in XML 1.0 (Second Edition)”http://www.w3.org/TR/xml-names/ for more information.

[TBD: Here I use URL format because it is widespread and users are more familiar with it. Note two confusing things about using URL as an XML namespace. First, the URLs are used as addresses. URL in XML namespace works mostly as a global unique identifier, not as an address. Second, reverse names order in URLs, for example, not the http://com.company.domain but http://domain.company.com .]

[TBD: Here I use the URL reverse order for the XML namespaces. Not for the full name but for the “before-solution” part of the name, for the first part of the name before first backslash. If you don’t want to use XML namespaces that works as the web addresses, consider to use the XML namespaces in the generic, nonreversible, sortable order.]

Compose the URLs for the XML namespaces in the reversal order like http://domain.company.com/solution/schemas/2009-05-15 and add the “com” part of URL.

<TargetNamespace> =:
http://<Domain>.<Company>.com/<SolutionShortName>/<ProjectShortName>/<Version

<Version> =:
<date>
[in YYYY-MM-DD format]

[TBD We considered using two version formats. One format is like 1.0.0.1, the format used for the .NET assemblies. See http://msdn.microsoft.com/en-us/library/51ket42z.aspx . The second version format uses a date. Use the first one only if you can implement strict versioning rules, and in this case you can create/have some version approval procedure. We use here the date format.]

5.1 Additional Rules for XML namespaces

Use the same XML namespace for all schemas in one project. Schemas in one project (with equal XML namespace) are differentiated by the root node names. Do not place the root node name inside the XML namespace.
Use the project creating date for the first versions of all schemas inside project.
Use the current date for the second and next versions.
Create the new version only if the old one is published to production (test) environment. Do not create new versions inside development cycles.
Use the YYYY-MM-DD date format to make the names “sortable”. Do not use MM/DD/YY format.
URLs are case-sensitive (Yes, they are!) but do not use the upper case letters in XML namespaces.

For example, for the MyCompany.MyDomain.MySolution solution and the MyProject project the XML namespace should be the http://mydomain.mycompany.com/mysolution/myproject/2009-05-15 for the first versions of all schemas, if this project was created in 2009-05-15.

6 BizTalk artifacts

6.1 Orchestrations, Schemas, Pipelines

Names of these artifacts appear together with BizTalk application name mostly everywhere. We don’t need to use composite names. Use simple names for the names.

Note: If you see one word in several places of the full name, consider this as a bad signal. Try to rethink the terms used in the names. I repeat this rule here, because exactly in the full names of orchestrations, schemas and pipelines you can frequently see the repetitive words.

6.2 Maps

<Map> =:
        <SourceSchema>_to_<DestinationSchema> [for one-to-one map]
       <SourceSchema1>_and_<SourceSchema2>_to_<DestinationSchema> [for two-to-one map]
           …

If it is possible do not change the schema names in the map name. If the map name is excessive long, cut the schema names, but use the same cut rule for all map names.

6.3 Ports

Ports are the primary artifacts of the BizTalk solution. But in contrast to orchestrations they are used through the BizTalk application boundaries in many places, that is why, we have to use the composite names for the ports like for assemblies.

Do we need to separate one and two-way ports and send and receive ports, for example be “R_” or “SR_” prefixes? Do they mix up in lookups or in lists? No. And answer to the first question is “No”, do not use prefixes in the port names.

Do we have to use the transport/protocol qualifiers in the port names, like .FILE or .SOAP? No. One port could use several protocols. Moreover port is on the upper level architecture than transport. But for dynamic ports the transport name can be the main part of the port name.

Do we have to use message type in the port names, like .Request? No. One port could work with several message types. But frequently the message type can be used for the port name.

Try to understand the main purpose of the port and use it in the name. For example, link the port name with transport for dynamic port; or link the port name with the partner name, or with message type.

For example, MyCompany.MyDomain.MySolution.MyPartner.

7 Orchestration Naming Conventions

7.1 Orchestration artifacts

Use the prefixes to differentiate the artifacts in the XLang expressions. These artifacts are not usual .NET objects. They are used in different language context and sometime they use different language syntax. Prefixes really help to work with these artifacts.

<RoleLink> =:
roleLink_ + <Name>

7.2 Orchestration artifact types

We can use one suffix the “_type” for all different types because different types are seen only in the different lists and never mixed. For instance, we can never see the port types together with message types.

<ArtifactType> =:
<ArtifactName> + “_type”

7.3 Port Shapes

where

	Send port	Receive port	Send-Receive (Solicit-Response) port	Receive-Send (Request- Response) port
prefix	S_+	R_+	SR_+	RS_+

For example, S_ OrderAck.

Notes:

The Port shapes are the real names, the names of the .NET objects. We can’t use spaces inside.
In the Orchestration view there are generic lists the “Ports” and the “Port types” that’s why we have to distinguish the ports with different Communication directions and pattern.

7.4 Orchestration Workflow Shapes

Problems with orchestration shapes:

· Shapes are too small to display long names (only 12-18 characters).

· We have to “hover mouse over” shape or click shape to show Properties window to "understand" this shape, to understand what message it is processed.

Useful features:

· Feel free to use the same names for different shapes and use spaces inside the shape names. Shape names are not the “real programming names”. In reality they are the descriptions (excluding the Port shapes names); they are used only for description and for nothing more.

· Icons on shapes give us the useful information. Do not repeat the “icon information” by words. For example, if we change a name of Construction shape from “Construct Input message” to “Input message” we get more clear definition because we have the Construct icon + name.

· Shape names are used only in Orchestration Editor (excluding the Port shapes names). We don’t have to force any rules to make the “well-sorted” names (it's the main purpose of the prefixes).

· Use a Group shape to add description to a group of related workflow shapes. Group shape will display as much text as you want. Group shapes add a lot of documentation value to the orchestration.

7.4.1 Rules for shapes

Purpose of the orchestration and the most of the shapes is in processing the messages. We can unambiguously describe the messages by the message type. That is why in the most cases using the message type names gives us the main information about this message. That is why in the most cases using the message type names as the shape names gives us the main information about this shape, about message flow, about whole orchestration processing. Send shape with name "OrderAck" means ... exactly!

· Whenever it is possible use the MessageType of the processed message as a shape name.

· Do not repeat the type of shape icon by word.

· Do not repeat words from external shape name into the internal, nested shape name.

· Feel free to use spaces inside the shape names.

· Feel free to repeat the shape names.

7.4.2 Rules for specific shapes

Construct, Receive, and Send:

= name of the processed message without “msg_” prefix.

For example, [OrderAck]

Note: it’s easy to set and maintain this name: just copy part of it from Properties/Messages Constructed to Properties/Name. For example, from “msg_OrderAck” copy “OrderAck”

Transform:

= “from “ + name of the Source message

For example, [from OrderAck]

Note: it’s easy to set and maintain this name: just copy it (or part of it) from Properties/Input Messages to Properties/Name. For example: From “msg_OrderAck” copy “OrderAck”

Assignment:

No strict rules, only advice:

Name it like the methods in classes. But cut a verb if it possible. Use “set” and “get” if it possible.

For example, [set OrderAck]

8 BizTalk artifact and project places

Consider this chapter as “out-of-scope”. I placed it here because we discussed the folder names here. The files placement is a separate and wealthy topic. Here are only main considerations.

· If the project is simple, place all files in one project.

· If we want to use some files for references from other projects, place these files in separate project.

· Place artifacts to different projects if these artifacts have different refactoring lifecycle. For example, the Niem standard schemas are never changed then place them to the separate project. The maps are changed more frequently than schemas and we could place schemas and maps to the separate projects.

· Don’t place the technology-specific schemas and maps away from the orchestration they used for. For example, for the SQL port we generate a (technology) schema and usually create the map to transform the original schema to this (technology) schema. Place these schema and map together with orchestration, not into the Schemas/Maps projects.

8.1 Project subfolders

In the BizTalk project folder add the subfolder the Tests. Use it to the unit tests. Inside the Tests create subfolders: In, Out, TestMessages.

For the BizTalk project with different artifact files you can add the solution folders the Schemas, Orchestrations, and Maps in case you have several schemas, orchestrations, and maps there.

9 Out of scope

Several BizTalk artifacts are out of scope this naming convention:

· BRE artifacts: Rule sets, Vocabularies, etc.

· BAM artifacts: Activities, Views, BAM Definitions, Tracking Profiles

· Parties, Role links

· Itineraries from the ESB Toolkit 2.0. I think the ESB Toolkit is a part of BizTalk 2009, because it is delivered in binary format, supported by Microsoft, has a lot of tooling, including Itinerary Designer.

See also

1. “Naming guidelines for the .NET Framework types” in MSDN [ http://msdn.microsoft.com/en-us/library/xzf533w0(VS.71).aspx ]

2. “Naming Convention” in Wikipedia [ http://en.wikipedia.org/wiki/Identifier_naming_convention ]

3. “Abbreviations” in MSDN [http://msdn.microsoft.com/en-us/library/141e06ef(VS.71).aspx ]

4. “Readability” in Wikipedia [See http://en.wikipedia.org/wiki/Typography ]

5. “Capitalization Styles” in MSDN [http://msdn.microsoft.com/en-us/library/x2dbyw72(VS.71).aspx ]

6. “Assembly Versioning” in MSDN [ http://msdn.microsoft.com/en-us/library/51ket42z.aspx ]

7. “Namespaces in XML 1.0” [ http://www.w3.org/TR/xml-names/ ]

8. URN in Wikipedia [ http://en.wikipedia.org/wiki/Uniform_Resource_Name ].

9. URL in Wikipedia [ http://en.wikipedia.org/wiki/URL ].

Appendix

Syntax

<Word> =:
[A-Za-z1-0]*

<CompositeWord> =:
<Word>.<Word>[.<Word>]

<{Any}ShortName> =: <ShortName>
[replace {Any} with any term. For example, term Solution creates a <SolutionShortName> term]

<SolutionsRootFolder> =: c:\Solutions

<Version> =:
<date>
[in YYYY-MM-DD format]

<Map> =:
<SourceSchemaShortName>_to_<DestinationSchemaShortName> [for one-to-one map]
<SourceSchemaShortName1>_and_<SourceSchemaShortName2>_to_<DestinationSchemaShortName> [for two-to-one map]
…

Instructions

Keep the names of some BizTalk artifacts in the lists, a list per solution. Create, at last, a list of the XML namespaces of all schemas. Keep them in one list; it forces developers to use naming convention, because inconsistencies in the names are visible inside these lists on the development stage.

1. Create a new name in the list below.

2. Copy it to the name property of the new BizTalk artifact.

XML namespace List

Port List

BizTalk: Debugging the BizTalk applications with the direct test send ports

Leonid Ganeline — Sun, 27 Apr 2008 22:54:27 GMT

" ...I have published an orchestration as a web service and consumed it from an aspx page. now, how do I see if I triggered the orchestration and how and where do I get the output (transformed message of the orchestration)? ..."

It is a very common question, how to debug the BizTalk applications.

Have you ever tried to use the BizTalk Orchestration Debugger? So painful experience. Never see the BizTalk developer who is using it in real work.

One simple decision is creating the direct test send ports. (http://blogs.msdn.com/kevin_lam/archive/2006/07/07/659214.aspx, http://geekswithblogs.net/LeonidGaneline/archive/2006/12/18/101541.aspx)

Test port is a port with the filter expression (http://msdn2.microsoft.com/en-us/library/aa578042.aspx, http://msdn2.microsoft.com/en-us/library/aa547903.aspx). This expression creates the subscription to the message you interested in.

All sent and received messages go through the MessageBox. When they pass MessageBox you can create additional subscription to these messages and intercept copies of these messages.

The subscription can be to the message type or to the publisher parameters like Receive port name.

This technique has great benefits:

The direct test send ports do not disturb the original message flow (excluding the rare case when all main subscriptions to the debugged message have got unenlisted).
They can be easily switch on/off.

As a matter of fact I use two methods on the regular basis. (The second one is the old good Trace.WriteLine(..) with DebugView.exe as a receiver of the debug output.) That means I always create the orchestrations with the Expression shapes (for Trace.WriteLine)) after all message receives/transformations, in the begin/end all branches, plus I create the test send ports for almost ALL messages.

Yes, for ALL output messages.

It is easy and fast to create the test send ports in the binding file without using UI.

And I always deploy these test ports to the production. It saved me a lot of time in deployment and maintain.

BizTalk: Accumulating messages in MessageBox, Lifespan of the messages

Leonid Ganeline — Wed, 14 Nov 2007 21:15:09 GMT

I hit this problem when were creating the orchestration to periodically poll the outer service. The orchestration was with loop inside.

The problem was I created the message inside this loop. And the instance of this message were not disposed after each cycle. The BizTalk does not have such functionality as explicit deleting the messages inside orchestration.

The orchestration was created for running long time without restart. That means that this orchestration was collecting enormous amount those indisposed messages. And all those messages were inside the MessageBox, of course. It was wrong!

After long and very interesting investigation the issue was resolved.

The source of the problem was in the transactional scope where I created the indisposable message. This scope was inside the loop. The message was recreated on each cycle of the loop.

It is interesting but a lifespan of the message recreated inside the transactional scope included in the loop and the message recreated outside the transactional scope in the loop is different. The first one is disposed only after closing the orchestration instance. The second one is disposed after last using. (I think so but could not find a hint in the BizTalk information sea.)

After discovering this the fix was simple. I moved the transaction scope outside of the loop.

BizTalk: The Endpoint model depreciates the Physical and Logical Ports

Leonid Ganeline — Thu, 27 Sep 2007 21:25:13 GMT

This is a 2nd article in the BizTalk Model serie (1st article is a "BizTalk messaging model"
http://geekswithblogs.net/LeonidGaneline/archive/2006/12/18/101541.aspx)

One of the difficult part of BizTalk to understanding is a Port. Particularly I mean, Physical Port and Logical Port.

Within BizTalk 2004 there are not such things as Physical Port and Logical Port. At first these terms started in several whitepapers, then they were "officially approved" in BizTalk 2006 documentation. Why the Port was divided to Physical and Logical one?
As we try to understand what is "the Port" mean, we have to separate different things under this name. We have one name but different meaning in different contexts. For example, we create Port as a Port shape in Orchestration but it is not a real Port, but something that links us with "real" Port. The "real", Physical Port works adapter and pipeline (or is it managing them? And this is a different story...) and exists as an independent artifact. By the way, looks like the "real" Port can create Instances (Or this is not a Port but some Messaging Agent? Or not a Port Instance but a Messaging Instance, isn't it? This is a different story as well...)
We might crate the Port into the Orchestration, it is a part of this Orchestration, isn't it? And we can create the Port independently with the BizTalk Explorer and the Administration Console. And we can see the Port list in the BizTalk Explorer and the Administration Console.
In this way, trying to differentiate the things in the Orchestration Editor and in the BizTalk Explorer were created terms the Logical Port and Physical Port. Logical Port it is a "Port-shape", and Physical Port it is a "Full-featured, big" Port in the BizTalk Explorer.
What's the problem with this model?

The problem is in terms ambiguity.

Why both a Physical Port and a Logical Port were called the Ports?
Maybe because the BizTalk developers try to make the link between Physical Port and Logical Port as an implicit link. If it works well, we don't need two different terms, all right. But it doesn't work, the BizTalk experience is that the link between Physical Port and Logical Port usually is an explicit link. For example, the publisher/subscriber pattern forces us to separate these terms.
Sometimes hiding the internal mechanism is a way to decrease the model complexity. But in this case it was opposite. Endeavour to hide this mechanism is a way to create additional unnatural terms, to distort the real BizTalk architecture.

I will try to show how BizTalk architecture could be if I use slightly different term set. This model is more easy to me. It would be great to get your feedback on this model.

BizTalk MessageBox (= BizTalk-bus):
This term is used without change. All messages first come to BizTalk MessageBox. Messages are published. Then messages are passed to subscriber. BizTalk MessageBox works as an Queue Server.

I'll change term the Port.
Port is the entity linking BizTalk with outer world. Send Port sends information to the outer world. Receive Port receives information from the outer world.
I define the outer world as the different protocols and applications (or more precisely the APIs).

Now this is a new term, the Endpoint. OK, this is not a new term :), we can find it in the SDK to describe the Adapter sample and several low-level things. But this term is carefully hidden by the BizTalk architects and documentation writers.

There are two kind of the Endpoints: Endpoint-Publisher (EP) and Endpoint-Subscriber (ES).
Ports and Orchestrations use the Endpoints to interact with MessageBox.
EP sends data to the MessageBox. EP sends all messages to the MessageBox, and it doesn't care of the consumers of the messages. If there are any at all.
It's the duty of the MessageBox to pass the data to the subscribers or suspend it.
ES receives the data from the MessageBox. It receives only subscribed data. A filter on the ES is created for this purpose. It takes responsibility of data passed from the MessageBox to the ES.
Endpoint always has only two sides.
One side is always linked with the MessageBox. The other is linked with a Port or an Orchestration.
Orchestration receives data from the MessageBox through the one kind of Endpoints (the ES) and sends data to the MessageBox through the second kind of Endpoints (the EP). The Orchestration has not any others links with the the Ports, the other Orchestrations, it only has links with the MessageBox through the Endpoints.

Now remember all those port-shape types in the Orchestration. How does it relate to this new model?
Say we create the Port shape in the Orchestration.
New Port: this creates an Endpoint: not Port at all;
New Configured port / Specify Later : the same;
New Configured port / Specify Now : this creates an Endpoint, then a new Port with the Filter linking Endpoint of the new Port and Endpoint of this Orchestration. Here the BizTalk creators were tried to hide all this mechanism with Ports and Endpoints. And this is helpful in this case.)
New Configured port / Direct : this creates an Endpoint without Filter.
New Configured port / Direct / Routing between Ports... : creating an Endpoint and a Filter linking it with an Endpoint of the Port (Routing between this Orchestration and Port)
New Configured port / Direct/ Self Correlating : this creates an Endpoint and switch on the instance subscription mechanism.
New Configured port / Direct / To receive(send) message to(from) other Orchestration... : creating an Endpoint and a Filter linking it with the Endpoint of other Orchestration (Routing between two Orchestration endpoints)

I'd say couple words about relationship of the old terms and new terms.
The old Port term is a secondary BizTalk artifact. From one side we can create the old port independently in the BizTalk Explorer. But the Orchestration Editor push us to look at the Port as an aftifact belonged to the Orchestration. And BTW, the port type could be saved only together with the Orchestration in assembly "belonged" to the Orchestration.
The old port is only the Port, the New Port is the combination of the Port and the Endpoint.
The old Orchestration can be coupled with Port, the new Orchestration does know nothing about Port.

Now the Port is the first-level artifact, independent of the Orchestration.
Now the Port and the Orchestration are the terms on the same level, with the same importance.

"Endpoint" model:
The high-level artifacts: MessageBox, Ports, Orchestrations.
Endpoints: the high-level artifacts should be connected only through the Endpoints and MessageBox.
Port has only one Endpoint. Orchestration has one Endpoint or more.
One side of an Endpoint is always linked with the MessageBox. The other side is linked with a Port or an Orchestration.
Endpoint-Publisher transmits data TO the MessageBox.
Endpoint-Subscriber transmits data FROM the MessageBox.

This is not a new model, it was implicitly used in many BizTalk articles.
Here I've tried to use this model explicitly.

I'm just thinking how easier would be explanations in the Kevin Lam's blog (MSFT) about Ports with new terms (see "Demystifying Direct Bound Ports" http://blogs.msdn.com/kevin_lam/archive/2006/04/18/578572.aspx).

...

BTW Seems this model depreciates the "Direct port" term too.

BizTalk "Get Last Message" pattern and orchestration

Leonid Ganeline — Mon, 06 Aug 2007 01:44:15 GMT

There is a case:

One part of the solution periodically publishes the data. This part is out of this article.

There are two kind of subscribers.

One kind of subscribers want to get the messages while they are published.

Second kind of subscribers pool this messages randomly:

undefined quantity of the subscribers request the undefined number times the LAST message in random interval.
they want to get the last input message only.

The usual publish-subscribe pattern doesn't work, because

the usual subscriber MUST receive ALL published messages. It cannot omit several and receive only others. (It can but it have to implement this logic by itself, it is not the case.)
After subscriber received the message, it cannot receive this message one more time. The message pops out of the query.
After one subscriber received the message, the other subscriber cannot receive the same message if it is not subscribed to the message. It is not a "random" , "one time", "randomly appeared/disappeared" subscriber.

The question is: How can I do this in "BizTalk style"?

The orchestration implements this "Get Last Message" pattern.

The orchestration has three parts.

One is for controlling this orchestration. It starts and stops the orchestration by the Control messages (Control START and Control STOP). That's all.

The second part is a part of the pattern. It receive all input messages (Trips_Canonical) in the Unifies Sequential convoy. Usually in the samples there are two receive shapes, one before and one inside the loop. I use the receive for the control message as an activate receive, it helps me to simplify the orchestration.

The third part is a part of the pattern too. It is compounded from the receive shape which requests the last message (Get Last Request message), and the send shape, which send this last message. The last message is the same as the input message, the only difference is in the value of the "ProcessName" element.

This sample is used in the content routed scenario that's why there are several interesting points.

all messages have one first level node, right under root node with name "ProcessName". This node holds the name of the message publisher. This helps route the messages with the same message type to different routes. For example, the input Trip_Canonical messages and the output Trip_Canonical have different values of this node, and this is the only difference.
the messages correlated by this "ProcessName" node (the cor_ApplicationName Correlation set).
I use the Decide shape to avoid to send the requested last message if I have nothing to send. But to get this project built I have to create the "foo" message.
I gather all receives under one direct port. This is an Ordered delivery=True port.

Maybe it worth to include the send port under this port too.

There are the Control messages:

START:

STOP:

Input message the "Trips_Canonical":

Get Last Request message:

Last message:

BizTalk: Architectural Tips: Monitoring orchestration

Leonid Ganeline — Thu, 15 Feb 2007 18:46:00 GMT

Note: this orchestration is used for learning purpose. To monitoring in the real life we'd rather use the BAM and BAS. I can see this orchestration in the real environment only as a "lightweight" tool when using an additional technology (BAM/BAS) is not worth the time.

Description:
This is a singleton orchestration for monitoring other BizTalk applications.
It can start/stop by the special "start/stop" message or whatever you want. It can work infinitely or on per day basis, or whatever you want.
It has a direct subscription to the monitoring activity, for example, to some message type, or to the the port name, or some parameters inside the messages.
It can create the log of this activity in a log file or collect this information to create the summary reports.

Implementation details of this sample:

An orchestration uses the sequential convoy to monitor the messages with specific "EDI document type" (it is an element of the all input messages). "Monitor" here means calculating of the message number.
It has "per EDI document type" instance. The different orchestration instances are started for different types of monitored messages.
The project references another project with monitored message (with schema of this message)
The orchestration has one receive port. It has to had parameter the "Ordered Delivery" as "True". I use parameters the "Binging" as "Direct" and the "Partner Orchestration Port" as "MessageBox", this simplify the relationship with others applications.
It uses one correlation set for two message types: for a monitored message and for a management message. The last one is used for output (trace) the current value of the monitored parameter and for stop the orchestration instance.

" LI <>

I can send you the project files by your request.

I would appreciate your feedback.

BizTalk messaging model

Leonid Ganeline — Tue, 19 Dec 2006 03:38:00 GMT

This is a 1st article in the BizTalk Model serie (2nd article is a "BizTalk: The Endpoint model depreciates the Physical and Logical Ports " http://geekswithblogs.net/LeonidGaneline/archive/2007/09/27/115663.aspx )

When I’ve started to master the BizTalk I start to build the different models for what is going on inside the BizTalk with messages. With each new iteration it was a different model.
I’ve begun with the well-known model:

Let’s name this model as a “Receiver-Sender model”.
This model is about the Senders and Receivers of the Messages.
Looks like the Receive Locations are the Receivers, the Send Ports are the Senders, and the Orchestrations are the Senders and the Receivers simultaneously. Right?
Wrong. All objects are the Senders and Receivers.
The Receive Locations receive the messages from the “Outer word” and send them to the Message Box.
The Send Ports receive the messages from the Message Box and send them to the “Outer word”.
The Orchestrations receive the messages from the Message Box and send them to the Message Box.
The Message Box receives the messages from the Receive Locations or the Orchestrations and sends them to the Send Ports and the Orchestrations.

There was a description of the Publisher-Subscriber model in the BizTalk Help. But the most of attention in the Help is to the Receiver-Sender model.

As I could understand, terms the Sender and Receiver confuse me.

To illustrate the ambiguity of Receive and Send the terms in the BizTalk context, I use the quotation from the book « Pro BizTalk 2006 » by George Dunphy and Ahmed Metwally, page 119:

« ... The left side shows the sender configuration, which consists of a static one-way send port to send the business document or message and a static one-way receive port to receive BizTalk Framework 2.0 delivery receipts. The right side shows the receiver configuration, which contains a static one-way receive port to receive the message ir business document and a dynamic one-way send port that... » (I’ve highlighted the terms.)

Ok. I have to say more about this ambiguity. Terms the Send and Receive means... the same. Are you send the message? Yes. And simultaneously this operation means to receive. This terms have meaning ONLY if they are used WITH source or target the operation. Do we send message? It is no sense in this question. We send message to target_object or from source_object. Do we receive this message? No sense! We receive it from source. Etc.

Subscribe and Publish the terms in BizTalk context always means "respectively the MessageBox". Terms Send and Receive do not have such context.

I don't wont to reject Send and Receive the terms completely. But they must be used in pair with the source or target of the operation, absolutely.

That's why in quotation above "the sender configuration" doesn't mean exactly what it should mean. In this context it would be better something like "the left configuration" because it was not SO confusing.

I’ve made some changes in the model:

Try to move Subscriber and Publisher the terms to the first place of the model.
This model is explicitly a “Message Box centered”.
The Publishers move the messages to the Message Box.
The Subscribers move the messages from the Message Box.
All actions are across the Message Box.

The Publishers move the messages with defined type (namespace + root_node) to the Message Box.
The Subscribers move from the Message Box the messages with parameters defined in the Filter expression.
In this case The Receive Locations and the Send Shapes are placed in the one group the Publishers, The Send Ports and the Receive Shapes are placed in the one group the Subscribers. It is how the wrong names could complicate the things.

How does the Binding fit this model?

I’ve got something like that.
The Binding = the Implicit Subscription.
This Subscription for Binding is created under scene, when the Publishers ID is used as the Filter expression in the most cases. The link via binding is not required; it can be created by an explicit filter expression.
“Binding” the term is used for describing the direct link between an Orchestration and a Port. As I understand this term was created for simplifying the model by hiding the additional work with creating the most frequently used operation, transferring the messages from one point exactly to the second one. Binding creates the One-to-One link when the messages from one Publisher must come to the one Subscriber and only to this Subscriber. Binding make a step from “pure” Publisher-Subscriber model where the Publishers and the Subscribers do know nothing about each other and have not to know.
Binding makes the model more complicated. It simplifies this operation but not a model.
The pure model is accessible in the BizTalk by a Direct port.

I’ve got the more generalized model:

Here I’ve pictured Orchestration in two parts. Before that the Send and Receive Shapes were in one Orchestration, which is wrong.
I have made the sizes of the Shapes and Port/Locations equal. I had sense that the Orchestrations should not be bigger then Ports.
I’ve added “Outer World”, why not? :)

This is the Publisher-Subscriber model.

Conclusions are simple:

The Receiver-Sender model complicates the things. Try do not use Receiver and Sender the terms for describing the message flow. Use Publisher and Subscriber the terms instead!
Binding should be learned on the second level of studying the BizTalk. It does not simplify the things if it is used to hide the Publisher-Subscriber model. Try to describe the message flow without Binding and things would be easy.

BizTalk: Service Instances: Orchestration and Messaging

Leonid Ganeline — Fri, 19 May 2006 04:25:00 GMT

IMHO
There should be the term "Port Instance" or something else.
In HAT we can see Service Instances in two main classes: Orchestration and Messaging.
The first one is quite understandable.
But the second one is not described in BTS Help.
As I can imagine it is the part of the port which created in "per-message" basis like Orchestration.
Now "Messaging" is the term for it. But Messaging could be attached to anything in BizTalk, it is too wide term.
But I don't mind any term. Now we DON'T have ANY term.
Nowhere in Help we could find description of this "Instance part of Port", we could only guess.
Now this term is used on "BizTalk developer team" level.

If it is not possible to hide it from BizTalk Users then BizTalk Users have to got this term, got to description of what's going on inside BizTalk with it.

PS
I don't like term the "Port Instance" but it is not the point.

BizTalk: Orchestration: Relationship of Receive port shape and Receive shape, Send port shape and Send shape

Leonid Ganeline — Sat, 18 Feb 2006 03:56:00 GMT

BizTalk: Orchestration: Relationship of Receive port shape and Receive shape, Send port shape and Send shape

I don't know would it be useful for somebody...
I'm constantly trying to understand the rules for different BizTalk tools.
For example, BizTalk Orchestration Designer:

Can I receive one message with the same type from different Receive port to one Receive shape?

No.

Pict.1

Can I receive the same message with the same type to several Receive shapes?

Yes.

Pict.2

Can I send the same message to the different Send ports?

No.

Pict.3

Can I send the different messages with the same type to several Send ports?

Yes.

Pict.4

Why there are these limits?

Quite obvious (??):

Inside Message Box the different messages with the same type transfered in the same manner. That's why for the messages with the same type permitted only one receive and one send endpoint.

Rule: "Links for the messages with the same type are always gathered in one received/send point on the port panel"

Try to link different Ports with the same message type with different Reseive shapes:

Pict.5.

We'v got an error #1:

...odx...: error X2214: you must specify at least one already-initialized correlation set for a non-activation receive that is on a non-selfcorrelating port
    : for example, mark the receive Activate property as True
    : or, mark the port Binding property as Direct and the Partner Orchestration Port as Self Correlating
    : or, check a correlation on the receive Following Correlation Sets property

Good.

If we've tried to mark (as recomended) the second Reseive shape with Activate property as True we got the different error #2:

...odx...: error X2071: an activatable receive must be the first executable statement in a service

ok. The second error message did not “correlate“ with first one. Sure, the error message #1 is not about my stupid changes :)

If I tried to mark the first Receive shape with “Initializing Correlation Sets“ and the second one with “Following Correlation Sets“, (It's not so stupid) I've got the error #3:

...odx...: error X2259: in a sequential convoy receive the ports must be identical
...Accounting.odx(353,22): could be 'R_Schema1'
...Accounting.odx(357,13): or 'R_Schema1_2'

Excelent! That's mean “my“ rule works!

If you have comments, please, give me a feedback!
Regards,

Leonid Ganeline
BizTalk Developer