BizTalk

BizTalk: xpath: How to work with empty and Null elements in Orchestration

Leonid Ganeline — Tue, 17 Nov 2009 23:55:12 GMT

The problem is with three Empty-Null cases.

Is it possible to separate all these cases in Expression shapes of the Orchestration?

For example, we have the record with <name> element, in such flawors:

case: "NonEmpty"

<ns0:People>
<ns0:Name>Name_0</ns0:Name>
<ns0:IsDependent>IsDependent_0</ns0:IsDependent>
..

case: "Empty"

  <ns0:People>
    <ns0:Name></ns0:Name>
    <ns0:IsDependent>IsDependent_0</ns0:IsDependent>
..

case: "OneTag"

<ns0:People>
<ns0:Name/>
<ns0:IsDependent>IsDependent_0</ns0:IsDependent>
..

case: "Null"

<ns0:People>

<ns0:IsDependent>IsDependent_0</ns0:IsDependent>
..

Is it possible to separate all these cases in Expression shapes of the Orchestration?

There is no information into the MSDN about this [http://msdn.microsoft.com/en-us/library/aa561906(BTS.10).aspx]

I tried to use the xpath() function in two variants, one with "string(xpath_expression)" second with "xpath_expression"

Expression Shape:

[

System.Diagnostics.Trace.WriteLine("== with string() ====================================================================");

var_xpathString = "string(/*[local-name()='Root' and namespace-uri()='http://MapTest.IncPerson']/*[local-name()='People' and namespace-uri()='http://MapTest.IncPerson']/*[local-name()='Name' and namespace-uri()='http://MapTest.IncPerson'])";

System.Diagnostics.Trace.WriteLine("[" + System.Convert.ToString(xpath (msg_SourceRoot, var_xpathString)) + "]");

if ( xpath (msg_SourceRoot, var_xpathString) == null)
{ System.Diagnostics.Trace.WriteLine("Name == null"); }

else if ( xpath (msg_SourceRoot, var_xpathString) == "")
{ System.Diagnostics.Trace.WriteLine("Name == Empty");}

else
{ System.Diagnostics.Trace.WriteLine("Name != null && Name != Empty"); }

System.Diagnostics.Trace.WriteLine("-- no string() --------------------------------------------------------------------");
var_xpathString = "/*[local-name()='Root' and namespace-uri()='http://MapTest.IncPerson']/*[local-name()='People' and namespace-uri()='http://MapTest.IncPerson']/*[local-name()='Name' and namespace-uri()='http://MapTest.IncPerson']";

System.Diagnostics.Trace.WriteLine("[" + System.Convert.ToString(xpath (msg_SourceRoot, var_xpathString)) + "]");

if ( xpath (msg_SourceRoot, var_xpathString) == null)
{ System.Diagnostics.Trace.WriteLine("Name == null"); }

else if ( xpath (msg_SourceRoot, var_xpathString) == "")
{ System.Diagnostics.Trace.WriteLine("Name == Empty"); }

else
{ System.Diagnostics.Trace.WriteLine("Name != null && Name != Empty");}

]

I.e. the "string(xpath_expression)" expression is used in the firs section, the"xpath_expression" is used in the second.

Result is:

"NonEmpty" == with string() ====================================================================
[Name_0]
Name != null && Name != Empty
-- no string() --------------------------------------------------------------------
[Microsoft.XLANGs.Core.Part+ArrayBasedXmlNodeList]
Name != null && Name != Empty

"Empty"== with string() ====================================================================
[]
Name == Empty
-- no string() --------------------------------------------------------------------
[Microsoft.XLANGs.Core.Part+ArrayBasedXmlNodeList]
Name != null && Name != Empty

"OneTag"== with string() ====================================================================
[]
Name == Empty
-- no string() --------------------------------------------------------------------
[Microsoft.XLANGs.Core.Part+ArrayBasedXmlNodeList]
Name != null && Name != Empty

"Null"== with string() ====================================================================
[]
Name == Empty
-- no string() --------------------------------------------------------------------
[]
Name == null

Conclusion:

* I cannot separate the cases "Empty" and "OneTag"

* I can separate the cases "Empty and "Null" with "xpath_expression", not with "string(xpath_expression)" expression
* "Null" case does not throw an exception.
* xpath expression inside string() works fine to get all three cases ("Empty", "OneTag", "Null") under one "if" statement. And here it returns only "" (Empty string).

BizTalk: Naming convention for the BizTalk solutions

Leonid Ganeline — Fri, 31 Jul 2009 17:44:57 GMT

Naming convention for the BizTalk solutions

Version 2.1, 2009-08-09

1 How to use this document

To accommodate this document for the specific solution:

· Create the real names in Appendix or create a separate Dictionary document. See Instructions in Appendix.

· Review the text, marked with [TBD] flag, and change it for your case.

2 General rules

The “Naming guidelines for the .NET Framework types” [ http://msdn.microsoft.com/en-us/library/xzf533w0(VS.71).aspx ] are used as a basis of this document. Also see the “Naming Convention” in Wikipedia [ http://en.wikipedia.org/wiki/Identifier_naming_convention ]

Names should be short, sortable, readable, discoverable, and self-described.

The main test for constructing name rule should be in questions:

· In what kind of places can I see this name?

· How easily can I work with the name in these places?

“Work” means to read, to find, and to understand sense without errors. Usually we work with names in lists. Think about these places like about namespaces in programming languages.

Short names:

Create as short as possible name.
Use abbreviations only in restricted cases. See article the “Abbreviations”http://msdn.microsoft.com/en-us/library/141e06ef(VS.71).aspx
Use prefixes and suffixes only to differentiate names.
If you see one word in several places of the full name, consider this as a bad signal. Try to redesign the terms used in the names.

Sortable:

Create “sortable” name. That means using more generic/important part of the name in the leftmost position. For example, prefer the name Folder_20090515 than the name Folder_05_15_2009.

Readable:

See http://en.wikipedia.org/wiki/Typography

Use the name case compatible with the well-known practices in programming languages/protocols with respect to upper and lower cases. For instance, the XML namespaces (URL) should be in lower case format, but the other names should be in the Pascal format. See the “Capitalization Styles” article [http://msdn.microsoft.com/en-us/library/x2dbyw72(VS.71).aspx ]
If the word with specific case is widely used in company, don’t force to change it to the Pascal format.
Decorate infixes, prefixes or suffixes with lower case and with undrscore. For example: TicketBatch_type, Source1 _and_Source2_to_Target, msg_MyRequest.

Discoverable:

The name should be discoverable. That means we easily should understand by the name of the artifact where we can find the artifacts or additional information about it. Names should link artifacts. Say, the schemas with XML namespace the http://domain.company.com/solution/project/2009-05-14 definitely should be in the project the Company.Domain.Solution.Project and in assembly the Company.Domain.Solution.Project.dll.

Self-described (Semantics):

Create name form the “business point of view” not form the “developer point of view”, especially the name exposed outside solution.
Don’t use generic terms in the names. Examples: Send, Receive, Service, Message, Transformation, Schema, Map, Orchestration, BizTalk.
Place a frequently used term into the shared dictionary with comment about where do use it and do not use.

Full, Short, and Composited names:

Several artifacts have the full names (composited names) and short names. Usually we can easily understand what name we use in the specific context.

· If we need to add one or more logical grouping into the name, use the composited names created with words separated with dot. In specific contexts the separation symbol can be different, like underscore, dash or backslash. Each <…ShortName> can be a composite name. Composite names can be used for each part of the full names, as Company, Domain, Solution, and Project. For example, you can use name the Schemas.Niem for the project short name.

3 General names

3.1 Syntax

3.2 Examples

· Solution name: short name – MySolution, full name - MyCompany.MyDomain.MySolution

· Project name: short composite name – Schemas.Niem; full name - MyCompany.MyDomain.MySolution.Schemas.Niem

4 Names inside BizTalk Solutions

4.1 Solutions, BizTalk applications

We use term the solution in the Visual Studio meaning, it is a name of the solution we see in the Solution Explorer window in the Visual Studio. Sometimes we mix solution and project terms but not in this document.

BizTalk application is named as solution.

For example, for the MySolution solution use the MyCompany.MyDomain.MySolution full name.

4.2 Projects, Assemblies, .NET namespaces

We use term the project in the Visual Studio meaning, it is a name of the project we see in the Solution Explorer window in the Visual Studio.

For example, for the MyProject project use the MyCompany.MyDomain.MySolution.MyProject project full name, the MyCompany.MyDomain.MySolution.MyProject assemblyname, and the MyCompany.MyDomain.MySolution.MyProject namespace.

We should separate the .NET namespaces and the XML namespaces. They are different things and used in different places. For brevity we use term the namespace exactly for the .NET namespace and always use the term the XML namespaces for the XML namespaces.

Note: After creating a new project, copy the project name property to the project properties the “Assembly Name” and the “Default Namespace”.

4.3 Folders

Use the predefined folder structure on all development machines and, if it is possible, on the Test and Product servers. This eliminates the errors and deployment effort because several BizTalk artifacts use the absolute file paths in configuration.

<SolutionsRootFolder> =: c:\Solutions

For example, the MyCompany.MyDomain.MySolution.MyProject project is placed in the c:\Solutions\MyCompany.MyDomain.MySolution\MyProject folder.

5 XML namespaces

Use very strict rules for the XML namespaces, because the XML documents expose the data interfaces to the outer world. The XML namespaces should follow the industry standards and the corporate standards. The exposed interfaces are immutable, why we should use versioning for XML namespaces. XML namespaces work as global unique identifiers for nodes of the XML documents.

We must use the URL or URN formats to XML namespaces. Feel free to use one of these standards. See the “Namespaces in XML 1.0 (Second Edition)”http://www.w3.org/TR/xml-names/ for more information.

[TBD: Here I use URL format because it is widespread and users are more familiar with it. Note two confusing things about using URL as an XML namespace. First, the URLs are used as addresses. URL in XML namespace works mostly as a global unique identifier, not as an address. Second, reverse names order in URLs, for example, not the http://com.company.domain but http://domain.company.com .]

[TBD: Here I use the URL reverse order for the XML namespaces. Not for the full name but for the “before-solution” part of the name, for the first part of the name before first backslash. If you don’t want to use XML namespaces that works as the web addresses, consider to use the XML namespaces in the generic, nonreversible, sortable order.]

Compose the URLs for the XML namespaces in the reversal order like http://domain.company.com/solution/schemas/2009-05-15 and add the “com” part of URL.

<TargetNamespace> =:
http://<Domain>.<Company>.com/<SolutionShortName>/<ProjectShortName>/<Version

<Version> =:
<date>
[in YYYY-MM-DD format]

[TBD We considered using two version formats. One format is like 1.0.0.1, the format used for the .NET assemblies. See http://msdn.microsoft.com/en-us/library/51ket42z.aspx . The second version format uses a date. Use the first one only if you can implement strict versioning rules, and in this case you can create/have some version approval procedure. We use here the date format.]

5.1 Additional Rules for XML namespaces

Use the same XML namespace for all schemas in one project. Schemas in one project (with equal XML namespace) are differentiated by the root node names. Do not place the root node name inside the XML namespace.
Use the project creating date for the first versions of all schemas inside project.
Use the current date for the second and next versions.
Create the new version only if the old one is published to production (test) environment. Do not create new versions inside development cycles.
Use the YYYY-MM-DD date format to make the names “sortable”. Do not use MM/DD/YY format.
URLs are case-sensitive (Yes, they are!) but do not use the upper case letters in XML namespaces.

For example, for the MyCompany.MyDomain.MySolution solution and the MyProject project the XML namespace should be the http://mydomain.mycompany.com/mysolution/myproject/2009-05-15 for the first versions of all schemas, if this project was created in 2009-05-15.

6 BizTalk artifacts

6.1 Orchestrations, Schemas, Pipelines

Names of these artifacts appear together with BizTalk application name mostly everywhere. We don’t need to use composite names. Use simple names for the names.

Note: If you see one word in several places of the full name, consider this as a bad signal. Try to rethink the terms used in the names. I repeat this rule here, because exactly in the full names of orchestrations, schemas and pipelines you can frequently see the repetitive words.

6.2 Maps

<Map> =:
        <SourceSchema>_to_<DestinationSchema> [for one-to-one map]
       <SourceSchema1>_and_<SourceSchema2>_to_<DestinationSchema> [for two-to-one map]
           …

If it is possible do not change the schema names in the map name. If the map name is excessive long, cut the schema names, but use the same cut rule for all map names.

6.3 Ports

Ports are the primary artifacts of the BizTalk solution. But in contrast to orchestrations they are used through the BizTalk application boundaries in many places, that is why, we have to use the composite names for the ports like for assemblies.

Do we need to separate one and two-way ports and send and receive ports, for example be “R_” or “SR_” prefixes? Do they mix up in lookups or in lists? No. And answer to the first question is “No”, do not use prefixes in the port names.

Do we have to use the transport/protocol qualifiers in the port names, like .FILE or .SOAP? No. One port could use several protocols. Moreover port is on the upper level architecture than transport. But for dynamic ports the transport name can be the main part of the port name.

Do we have to use message type in the port names, like .Request? No. One port could work with several message types. But frequently the message type can be used for the port name.

Try to understand the main purpose of the port and use it in the name. For example, link the port name with transport for dynamic port; or link the port name with the partner name, or with message type.

For example, MyCompany.MyDomain.MySolution.MyPartner.

7 Orchestration Naming Conventions

7.1 Orchestration artifacts

Use the prefixes to differentiate the artifacts in the XLang expressions. These artifacts are not usual .NET objects. They are used in different language context and sometime they use different language syntax. Prefixes really help to work with these artifacts.

<RoleLink> =:
roleLink_ + <Name>

7.2 Orchestration artifact types

We can use one suffix the “_type” for all different types because different types are seen only in the different lists and never mixed. For instance, we can never see the port types together with message types.

<ArtifactType> =:
<ArtifactName> + “_type”

7.3 Port Shapes

where

	Send port	Receive port	Send-Receive (Solicit-Response) port	Receive-Send (Request- Response) port
prefix	S_+	R_+	SR_+	RS_+

For example, S_ OrderAck.

Notes:

The Port shapes are the real names, the names of the .NET objects. We can’t use spaces inside.
In the Orchestration view there are generic lists the “Ports” and the “Port types” that’s why we have to distinguish the ports with different Communication directions and pattern.

7.4 Orchestration Workflow Shapes

Problems with orchestration shapes:

· Shapes are too small to display long names (only 12-18 characters).

· We have to “hover mouse over” shape or click shape to show Properties window to "understand" this shape, to understand what message it is processed.

Useful features:

· Feel free to use the same names for different shapes and use spaces inside the shape names. Shape names are not the “real programming names”. In reality they are the descriptions (excluding the Port shapes names); they are used only for description and for nothing more.

· Icons on shapes give us the useful information. Do not repeat the “icon information” by words. For example, if we change a name of Construction shape from “Construct Input message” to “Input message” we get more clear definition because we have the Construct icon + name.

· Shape names are used only in Orchestration Editor (excluding the Port shapes names). We don’t have to force any rules to make the “well-sorted” names (it's the main purpose of the prefixes).

· Use a Group shape to add description to a group of related workflow shapes. Group shape will display as much text as you want. Group shapes add a lot of documentation value to the orchestration.

7.4.1 Rules for shapes

Purpose of the orchestration and the most of the shapes is in processing the messages. We can unambiguously describe the messages by the message type. That is why in the most cases using the message type names gives us the main information about this message. That is why in the most cases using the message type names as the shape names gives us the main information about this shape, about message flow, about whole orchestration processing. Send shape with name "OrderAck" means ... exactly!

· Whenever it is possible use the MessageType of the processed message as a shape name.

· Do not repeat the type of shape icon by word.

· Do not repeat words from external shape name into the internal, nested shape name.

· Feel free to use spaces inside the shape names.

· Feel free to repeat the shape names.

7.4.2 Rules for specific shapes

Construct, Receive, and Send:

= name of the processed message without “msg_” prefix.

For example, [OrderAck]

Note: it’s easy to set and maintain this name: just copy part of it from Properties/Messages Constructed to Properties/Name. For example, from “msg_OrderAck” copy “OrderAck”

Transform:

= “from “ + name of the Source message

For example, [from OrderAck]

Note: it’s easy to set and maintain this name: just copy it (or part of it) from Properties/Input Messages to Properties/Name. For example: From “msg_OrderAck” copy “OrderAck”

Assignment:

No strict rules, only advice:

Name it like the methods in classes. But cut a verb if it possible. Use “set” and “get” if it possible.

For example, [set OrderAck]

8 BizTalk artifact and project places

Consider this chapter as “out-of-scope”. I placed it here because we discussed the folder names here. The files placement is a separate and wealthy topic. Here are only main considerations.

· If the project is simple, place all files in one project.

· If we want to use some files for references from other projects, place these files in separate project.

· Place artifacts to different projects if these artifacts have different refactoring lifecycle. For example, the Niem standard schemas are never changed then place them to the separate project. The maps are changed more frequently than schemas and we could place schemas and maps to the separate projects.

· Don’t place the technology-specific schemas and maps away from the orchestration they used for. For example, for the SQL port we generate a (technology) schema and usually create the map to transform the original schema to this (technology) schema. Place these schema and map together with orchestration, not into the Schemas/Maps projects.

8.1 Project subfolders

In the BizTalk project folder add the subfolder the Tests. Use it to the unit tests. Inside the Tests create subfolders: In, Out, TestMessages.

For the BizTalk project with different artifact files you can add the solution folders the Schemas, Orchestrations, and Maps in case you have several schemas, orchestrations, and maps there.

9 Out of scope

Several BizTalk artifacts are out of scope this naming convention:

· BRE artifacts: Rule sets, Vocabularies, etc.

· BAM artifacts: Activities, Views, BAM Definitions, Tracking Profiles

· Parties, Role links

· Itineraries from the ESB Toolkit 2.0. I think the ESB Toolkit is a part of BizTalk 2009, because it is delivered in binary format, supported by Microsoft, has a lot of tooling, including Itinerary Designer.

See also

1. “Naming guidelines for the .NET Framework types” in MSDN [ http://msdn.microsoft.com/en-us/library/xzf533w0(VS.71).aspx ]

2. “Naming Convention” in Wikipedia [ http://en.wikipedia.org/wiki/Identifier_naming_convention ]

3. “Abbreviations” in MSDN [http://msdn.microsoft.com/en-us/library/141e06ef(VS.71).aspx ]

4. “Readability” in Wikipedia [See http://en.wikipedia.org/wiki/Typography ]

5. “Capitalization Styles” in MSDN [http://msdn.microsoft.com/en-us/library/x2dbyw72(VS.71).aspx ]

6. “Assembly Versioning” in MSDN [ http://msdn.microsoft.com/en-us/library/51ket42z.aspx ]

7. “Namespaces in XML 1.0” [ http://www.w3.org/TR/xml-names/ ]

8. URN in Wikipedia [ http://en.wikipedia.org/wiki/Uniform_Resource_Name ].

9. URL in Wikipedia [ http://en.wikipedia.org/wiki/URL ].

Appendix

Syntax

<Word> =:
[A-Za-z1-0]*

<CompositeWord> =:
<Word>.<Word>[.<Word>]

<{Any}ShortName> =: <ShortName>
[replace {Any} with any term. For example, term Solution creates a <SolutionShortName> term]

<SolutionsRootFolder> =: c:\Solutions

<Version> =:
<date>
[in YYYY-MM-DD format]

<Map> =:
<SourceSchemaShortName>_to_<DestinationSchemaShortName> [for one-to-one map]
<SourceSchemaShortName1>_and_<SourceSchemaShortName2>_to_<DestinationSchemaShortName> [for two-to-one map]
…

Instructions

Keep the names of some BizTalk artifacts in the lists, a list per solution. Create, at last, a list of the XML namespaces of all schemas. Keep them in one list; it forces developers to use naming convention, because inconsistencies in the names are visible inside these lists on the development stage.

1. Create a new name in the list below.

2. Copy it to the name property of the new BizTalk artifact.

XML namespace List

Port List

Book “SOA Patterns with BizTalk Server 2009” by Richard Seroter, review

Leonid Ganeline — Wed, 29 Jul 2009 01:10:57 GMT

Book “SOA Patterns with BizTalk Server 2009” by Richard Seroter, review

Here is a page from publisher ([PACKT] Publishing) http://www.packtpub.com/soa-patterns-with-biztalk-server-2009/book

Questions:

Who is the reader of this book? What is the knowledge and experience level of the reader? How good is the book structure?

I think the reader should be a seasoned BizTalk developer. It is not for an entry level developer.

This book is for architects, but for architects with wealthy knowledge of the BizTalk. I would suggest it for a senior BizTalk developers, which is equal to an Integration Architect title.

You can read the first three and the last four chapters of this book only for a quick review of your knowledge.

I highly recommended the chapters from 4th to 7th. They are from Richards’s wealthy experience. They are the heart and soul of the book. I would like to see much more such interesting things, maybe in next edition of this book?

Is it about BizTalk 2009 or about BizTalk?

Chapters from 9th to 12th are about BizTalk 2009 features and tools. Other chapters are more common and do not depend on the last version. They are more than that, better than that.

Is it about SOA Patterns?

Yes, it is.

Is it the “recipe” book?

The book contains a few good recipes, but indeed it is not a recipe book.

Is it the button-to-button book?

No, luckily it is not.

How is the book covering the topics?

Chapter	Audience	Level (1-5)	Grade (1-5)
Chapter 1: Building BizTalk Server 2009 Applications	Architect Developer	3 1	3 1
Chapter 2: Windows Communication Foundation Primer	Developer	1	1
Chapter 3: Using WCF Services in BizTalk Server 2009	Developer	3	3
Chapter 4: Planning Service-Oriented BizTalk	Architect	3	4
Chapter 5: Schema and Endpoint Patterns	Developer, Architect	3	5
Chapter 6: Asynchronous Communication Patterns	Developer	4	5
Chapter 7: Orchestration Patterns	Developer, Architect	5 3	4 4
Chapter 8: Versioning	Developer	3	3
Chapter 9: New SOA Capabilities in BizTalk Server 2009: WCF SQL Server Adapter	Developer	3	3
Chapter 10: New SOA Capabilities in BizTalk Server 2009: UDDI Services	Developer	3	2
Chapter 11: New SOA Capabilities in BizTalk Server 2009: ESB Guidance 2	Developer	3	3
Chapter 12: What's Next

Where Level:

1 – developers with entry level knowledge of BizTalk and no working experience

2 – developers with entry level knowledge of BizTalk and small working experience

3 – developers with fair level knowledge of BizTalk and fair working experience

4 – developers with expert level knowledge of BizTalk and fair working experience

5 – developers with expert level knowledge of BizTalk and expert working experience

I have to say, that several parts of this book “must be read” by every BizTalk developer. I insist these parts MUST be an essential part of the BizTalk Documentation from the early start and it is a shame of Microsoft these chapters are not included in the BizTalk Documentation. For example, the Schema Patterns, Chapter 5, how could developers work with Web-services without basic knowledge about basic principles of serializing schemas to .NET classes?

Sometimes author jumps from really interesting discussions about patterns to examples of how to implement it in “too much details” fashion.

When I ticked on the chapters with 2 or 1 grade, I thought “I cannot find any reason to include this chapter in the book. Here Richard presents only the general information but nothing from his experience. Geners description and general examples, but I would prefer to know the expert opinions, expert arguments, expert views, expert pros and cons.” I understand why these chapters are in the book. However, I strongly believe that the book worth reading if the author express his/her opinions in the text.

For Chapter 11 about ESB I would suggest to see the webcast by Richard Seroter “A look at the ESB Toolkit 2.0 in BizTalk Server 2009” http://cloudtv.cloudapp.net/ViewWebcast.aspx?webcastid=2521553277324634479. It is kind of the up-to-date version of this Chapter.

Conclusion

Pros:

“SOA Patterns with BizTalk Server 2009” book contains several very interesting topics.

Book contains an expert point of view.

Book also covers several useful SOA patterns implemented in/with BizTalk Server.

The best part of the book is not only about “how” but about “why” as well.

Cons:

Apparently, the author includes some chapters just to pad the book. But the usable volume is still more than a half. This is good proportion. Yes, it is a good proportion. Usually books of this kind have smaller “performance index”.

Conclusion

This book would be very helpful for the Integration Architects and BizTalk Developers.

Author is one of the most respectful BizTalk experts in the world and his point of view is always interesting.

Must have for every BizTalk Developer.

Highly Recommended

BizTalk WCF-BasicHttp adapter issue with http://www.w3.org/2001/XMLSchema namespace

Leonid Ganeline — Tue, 30 Jun 2009 04:31:51 GMT

BizTalk WCF-BasicHttp adapter issue with http://www.w3.org/2001/XMLSchema namespace

I am using the WCF-BasicHttp adapter to consume outer Web-service and getting the error (actually if I have retry>0, I am getting Warning first. See below:

Event Type: Warning
Event Source: BizTalk Server 2006
Event Category: BizTalk Server 2006
Event ID: 5743
Date:  ...
Time:  ...
User:  ...
Computer: ...
Description:
The adapter failed to transmit message going to send port "..." with URL "...". It will be retransmitted after the retry interval specified for this Send Port. Details:"Unable to read the stream produced by the pipeline.
Details: The value 'd:date' is invalid according to its schema type 'http://www.w3.org/2001/XMLSchema:QName' - 'd' is an undeclared namespace. ".
...

or (After retry the previous message the retry limit for this port)

Event Type: Error
Event Source: BizTalk Server 2006
Event Category: BizTalk Server 2006
Event ID: 5754
Date:  ...
Time:  ...
User:  ...
Computer: ...
Description:
A message sent to adapter "FILE" on send port "..." with URI "..." is suspended.
Error details: Unable to read the stream produced by the pipeline.
Details: The value 'd:date' is invalid according to its schema type 'http://www.w3.org/2001/XMLSchema:QName' - 'd' is an undeclared namespace.
MessageId: ...
InstanceID: ...
...

Very srange. When I tried the same Web-service with SoapUi, the result of the verification of the Response message is similar.

line 8: Invalid xsi:type qname: 'd:string' in element ...@http://...

What is wrong with response message?

Response Message:

<e:Envelope xmlns:d="http://www.w3.org/2001/XMLSchema" xmlns:i="http://www.w3.org/2001/XMLSchema-instance" xmlns:wn1="..." xmlns:wn2="..." xmlns:wn0="..." xmlns:e="http://schemas.xmlsoap.org/soap/envelope/">
   <e:Body>
      <wn2:ActionResponse>
         <wn2:MyResponse i:type="wn2:MyResponseType">
            <wn2:Detail i:nil="true"/>
            <wn2:MyMsg i:type="wn2:ArrayOfMyMsgType">
               <wn2:MyMsgType i:type="wn2:MyMsgType">
                  <wn2:MsgDescriptionText i:type="d:string">...</wn2:MsgDescriptionText>
                  <wn2:MsgIdentificationID i:type="d:string">...</wn2:MsgIdentificationID>
                  <wn2:MsgSeverityLevelCode i:type="wn2:MsgSeverityLevelType">...</wn2:MsgSeverityLevelCode>
                  <wn2:MsgSourceText i:nil="true"/>
               </wn2:MyMsgType>
            </wn2:MyMsg>
         </wn2:MyResponse>
      </wn2:ActionResponse>
   </e:Body>
</e:Envelope>

Investigation discovered that the issue is in the namespace prefix "d". If I change it to the "xsd", the error disappeared.

Questions:

How to work around this?
Is it the error in the response message?
Is it an error in the WCF-BasicHttp adapter?

How to work around this issue?
I changed the WCF-BasicHttp port for the old SOAP port and SOAP port works fine.

Is it the error in the response message?
I didn't find any limits for the "http://www.w3.org/2001/XMLSchema" namespace and for prefixes for it. The only limit is "don't use the prefixes, started with 'xml' letters". It is the common practice to use "xsd" prefix for the "http://www.w3.org/2001/XMLSchema" namespace. But it is not the rule.
Seems the WCF-BasicHttp adapter is "too smart".

Is it an error in the WCF-BasicHttp adapter?
possible...

BizTalk 2009: notorious Microsoft ADO MD.NET 9.0 and 10.0

Leonid Ganeline — Thu, 09 Apr 2009 01:56:33 GMT

BizTalk 2009: notorious Microsoft ADO MD.NET 9.0 and 10.0

When I was installing the BizTalk 2009 the Setup asked these prerequisite components:

Microsoft ADO MD.NET 9.0 and

Microsoft ADO MD.NET 10.0

I did not have Internet access and I had to manually search for these components.

Seems it is not a simple googling. :)

After painful search I've found them here:

1) ADO MD.NET 9.0:

Page "Feature Pack for Microsoft SQL Server 2005 - November 2005" - http://www.microsoft.com/downloads/details.aspx?FamilyID=d09c1d60-a13c-4479-9b91-9e8b9d835cdc&displaylang=en

[- follow PageDown to]

Microsoft ADOMD.NET

...

X86 Package (SQLServer2005_ADOMD.msi) - 3302 KB

X64 Package (SQLServer2005_ADOMD_x64.msi) - 5430 KB

IA64 Package (SQLServer2005_ADOMD_ia64.msi) - 6906 KB

2) ADO MD.NET 10:

Page "Microsoft SQL Server 2008 Feature Pack, October 2008" - http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=228de03f-3b5a-428a-923f-58a033d316e1

[- follow PageDown to]

Microsoft ADOMD.NET

...

X86 Package(SQLSERVER2008_ASADOMD10.msi) - 4312 KB

X64 Package (SQLSERVER2008_ASADOMD10.msi) - 9263 KB

IA64 Package(SQLSERVER2008_ASADOMD10.msi) - 6776 KB

As you can see there is a bad inconsistency in the names!

Hope this article saves you several minutes.

BizTalk: What features we would like to see in future releases

Leonid Ganeline — Wed, 01 Apr 2009 21:08:44 GMT

BizTalk: What features we would like to see in future releases

After the MVP Summit [http://geekswithblogs.net/LeonidGaneline/archive/2009/03/09/microsoft-global-summit-mvp-most-valuable-professional-2009-pictures.aspx] we, the BizTalk MVPs, have been asked to give feedback to the product team on what features we’d like to see in future releases.

I created the list of features and separated it into parts. One is from the global, crazy things than never be implemented. But why don’t imagine? Second one is from the small things.

Let start from the big, crazy things:

More flexibility in the deployment and licensing.

What I mean:

1) Why don't move the BizTalk as a product to cheap segment? (cheap? No, let's talk about bigger)

a) BizTalk + SQL Express on one box. Then creating a farm of the BizTalk applications would be easier. Load Balancing is moved outside, before the farm. Maybe create a separate product the "BizTalk load balancer"?

b) Moreover, ship the preconfigured VHD with BizTalk + SQL Express + Windows Server. With possibility to create the farm from these VHDs. (Not so easy because now we have strict limit to the SQL and BizTalk server names, and that limit should be fixed anyway.)

2) Why don't move to the expensive segment?

a) Shift to the Windows HPC? What about this? One time the BizTalk was pioneered as the first enterprise level product by Microsoft, created on .NET. Using the HPC methods to get rid of SQL completely, perform all persistency in memory to get super-low latency product.

3) Reiterate the architecture with all spectrum new Microsoft technologies. Is it possible to extract the MessageEngine, the OrchestrationEngine, and the UnifiedInputOutput Engine to separate products?

4) At last extract the BRE to separate product or free library! Now it is dead. Give BRE a shock treatment.

5) Redesign the error processing for different stages of the message processing. Now there are different approaches are used in many places or nothing at all.

6) Extract the LOB Adapters to separate product on the “buy if you need it” basis or to the free library. Create the Internet store and sell adapters for small money to everyone not only to the buyers of the BizTalk. Sell them not in packages but per adapter.

Now the small things:

Many years there were no improvements in very basic things, the things the developers are using in everyday tasks. I mean: adapters: File, SQL; pipelines; schema and map editors; additional shapes in Orchestrations, etc.

1. Adapters:

a. File adapter (for example): add the possibility to use the file names to order delivery (process files in the sorted order, many variations J ); implement the file receive filters with regEx on the file names; add possibility to copy files to other folder after consuming (to make some sort audit/resubmit storage, in forums this question is repeated and repeated, "How we could make a copy of the original files?")

b. SQL adapter: OMG, there should be many changes in real-time functions and much more improvements in the design-time appearance. I'd prefer to use not the standard Wizard to generate SQL port artifacts but some sort rich-UI window to set up different stages of artifact creation and the real-time parameters.
I am sure the brainstorm by BizTalk MVP should generate a lot of good ideas.

c. Additional adapters: like Task Scheduler; Nope; Fan-out/Garbage (to consume messages without subscribers in the pub/sub application); PowerShell (for additional processing of the input/output messages). Brainstorm this! Buy these adapters from the third-parts, unify them and include to the standard package.

d. Please, make InfoPath integration with BizTalk easy as hell!!!! (Isn't it a prime purpose of InfoPath??? To integrate the BizTalk with human?)

2. Pipelines (pipeline components): where are? the xml <-> PDF ; zip/unzip ; map (yeah! what is wrong with it?); different coders/decoders; symbol (regex) replacement (for EDI processing it MUST be!); PowerShell (for additional processing of the input/output messages); "create message" pipeline; Excel/Word ; Html ; simple EDI (to very basic EDI processing); promote properties from the text; etc. . Brainstorm this!

3. Add the error processing to the standard interfaces of the custom pipeline components.

4. Transmitting raw/binary data. Frequently we only use the BizTalk to transfer and route the documents (messages) without transformation. Now it is the second-class citizen scenario.

5. Orchestrations: additional shapes as: CreateMessage !!!; throwEvent; Pipeline transformation (?).
Make shapes resizable, otherwise we should use odd methods to make the Orchestration more descriptive. How we could show Orchestration to business people if we cannot add enough text to shapes???? Add color to shapes.
Make the Expression Editor window resizable, dockable, fully InteliSensed, etc. !!!!!!!!!!!!!!!!!!

6. Schema editor: Use the schema node icons for more useful information, definitely, for cardinality (say, different colors for "max occurs = *", for "min occurs = 0"). Maybe add separate window for the referenced (included) schemas?

7. Map editor: add functoids [IfElse]... Make improvements to the Script functoid!!! Separate to InlineScript; ExternalScript (ExternalMethod?); XsltScript...; make InteliSense to the Inline Script and Xslt script windows. Use the source/target node icons to more usefull information. See http://geekswithblogs.net/LeonidGaneline/archive/2005/12/23/64004.aspx - it is an old list but still actual.

8. Ports: Please, make ports the first-class artifact as an Orch. Why we should use the orchestration assembly just to store the port type??? Why do we use the "logical port" term in Orchestration editor? It is not a port, it is just an endpoint. Orchestration has the endpoints and the Port has the endpoints. My perception it should be big redesign of the "port part" of the BizTalk architecture. Right now it is a mix of adapters, pipelines, maps, etc. without strict regulation, strict design. This mix is very static, could not be used in structured way to improve the development.

The question to us was: 1) What was/is THE GOOD stuff about the area? 2) What is just plain BAD? 3) What is UGLY about the feature area e.g. if you were developing this feature what would you do to improve it.

I am sorry but I used the same words. To me “UGLY” and “BAD” are just synonyms for ”rudimentary implemented” and “implemented in beta version”.

1) Monitoring, operations. Now it is UGLY. No real real-time :). See how it works on HPC Windows, for example.

2) Debugging is plain BAD.

3) BRE is good, but EVERYTHING for development BR, everything is UGLY. Development, testing-debugging (come on, have somebody seen the test text window in Composer? the text inside it????), deploying. BRE now is good, very good to sell the BizTalk to business people, and BRE is just pain in ass for developers. Examples?
http://geekswithblogs.net/LeonidGaneline/archive/2006/12/27/102156.aspx
http://geekswithblogs.net/LeonidGaneline/archive/2006/10/16/94245.aspx
Now as a rule of thumb, I use the BRE for POC projects and never for real projects.

4) To me BAM is kind of partly implemented. Run-time is very good. Design-time is BAD.

5) ESB. IMHO if the BizTalk gets the Web-service (or metadata) repository, it has ALL ESB functionality and could be named ESB to business people. MVPs discussed this many times on the Summit.

6) Of course, low latency must be implemented.

Microsoft Global Summit MVP [Most Valuable Professional] 2009: Pictures

Leonid Ganeline — Mon, 09 Mar 2009 17:30:49 GMT

Hi All,

I've made pictures on the

Microsoft Global Summit MVP [Most Valuable Professional] 2009 March 1-4, Seattle, Redmond.
Mostly the BizTalk MVPs.

http://public.fotki.com/leogan/cities_and_countries/usa/2009-microsoftgloba/

Steve Ballmer, many vice presidents, Microsoft Fellows, developers from different product teams.

It was fun.

BizTalk: Debugging: Why is it so difficult?

Leonid Ganeline — Thu, 09 Oct 2008 21:12:08 GMT

Sometimes in the BizTalk forums such questions appear.

"...I have been using event log to test variables. I was hoping there was a way to watch it process in VS like you can debug a forms app."

There is a difference in the usual app and in the "BizTalk app".

When you start usual app it is exactly this app and you could debug it. When you "start" the BizTalk app that could means *many* *instances* of this BizTalk app could works simultaneously.

With BizTalk you have an intermediate layer, the host. This host (actually it is a host instance but it doesn't matter for this case) manages the app *instances*. It could start many app instances simultaneously and then dehydrate, rehydrate, stop, abort them without our intervention.

Ideally we need to attach a debugger to exactly one app instance (which one?) or create a debugger working with many app instances simultaneously.

It is one of the source of the complexity of debugging the BizTalk apps.

BTW What is a BizTalk application at all?

Please, let me know, what do you think.

BizTalk: Patterns: discussion "BizTalk, From Hub/Spoke to ESB"

Leonid Ganeline — Wed, 03 Sep 2008 05:13:42 GMT

There is an interesting discussion "BizTalk, From Hub/Spoke to ESB" in the MSDN BizTalk forum: [http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=3478379&SiteID=1]

BizTalk: Interview: Skills measurment

Leonid Ganeline — Thu, 10 Jul 2008 21:40:39 GMT

This is a Part 4 of the Interview articles.
Part 1: "BizTalk 2004, Questions for interview without answers" http://geekswithblogs.net/LeonidGaneline/archive/2006/05/22/79267.aspx
Part 2: "BizTalk interview questions and principle" http://geekswithblogs.net/LeonidGaneline/archive/2007/07/03/113663.aspx
Part 3: "WCF: Questions for studing and interview" http://geekswithblogs.net/LeonidGaneline/archive/2008/01/07/wcf-questions-for-studing-and-interview.aspx

Please, mark your skills in the table.

I completely understand that nobody ever tried ALL the BizTalk features.

Name                                                      Number of the Projects      Months             Skills Mark (1..10 or A,B...)

artifacts:

applications
ports, groups, locations
FTP
SOAP
HTTP
SQL
WCF
SMTP, POP3
MSMQ
SharePoint
LOB: EDI base, Siebel, etc. (name them)
pipelines

pipeline components
schemas
maps
database functoids (or custom DB mapping)
Xslt
Xslt templates
orchestrations
transactions
atomic scope
long-running
compensation scope
exception scope
persistent points
correlations
helper .NET classes
SQL data bases (with relation to the BizTalk)
hosts, instances
adapters
parties
EDI, AS2 system
policy, rules, vacabularies
BAM
BAS

Development:

orchestrations
port-to-port
helper .NET classes
custom adapters
custom pipeline components
custom functoids
rules…
BAM
BAS
ESB

Debugging Tuning-Up:

VS

Nunit
BizUnit
LoadGen
stress testing
performance metrics
different utilities (name them)
multi-server debugging
MOM
HAT
Adm.Console
PerfMon

Deployment:

BTSDeploy
BTSTask
MSI, Binding
multi-server deployment
clustered MessageBox
WMI scripts

Development techniques:

   - correlation
   - convoy parallel
   - convoy sequential unified
   - convoy sequential non unified
   - long-running transaction
   - transaction compensation
- FIFO
   - send data to SQL
   - receive data from SQL
   - data validating
- error handling in the BizTalk applications
- BizTalk to/from WS/HTTP apps
* test-driven development
* .NET development tools

Programming skills:

* test-driven programming
* VS.NET
* script languages, utils
* SQL: database normalization, programming
Web-programming
low-end programming

Several questions (sorry, no more than 2-3 min for answer):

How do you study BizTalk? what are the main sources of the information?
Describe creating the full-featured BizTalk application. Just the steps.
Describe debugging the full-featured BizTalk application.
Describe deploying the full-featured BizTalk application.
Monitoring the BizTalk application:

Skills Mark Legend:

1 can describe purpose and functionality
2 made Sample(s)
3 used in one project
4 used in one project recently
5 used in several projects
6 used advanced functionality
7 can show my answers in forums and articles in blog
8 used API (in code)
9 can teach other developers, can show pro and cons
OR
A know something
B can use in standard cases
C can use advanced features, API
D expert