- Architecture

BizTalk: Naming convention for the BizTalk solutions

Leonid Ganeline — Fri, 31 Jul 2009 17:44:57 GMT

Naming convention for the BizTalk solutions

Version 2.1, 2009-08-09

1 How to use this document

To accommodate this document for the specific solution:

· Create the real names in Appendix or create a separate Dictionary document. See Instructions in Appendix.

· Review the text, marked with [TBD] flag, and change it for your case.

2 General rules

The “Naming guidelines for the .NET Framework types” [ http://msdn.microsoft.com/en-us/library/xzf533w0(VS.71).aspx ] are used as a basis of this document. Also see the “Naming Convention” in Wikipedia [ http://en.wikipedia.org/wiki/Identifier_naming_convention ]

Names should be short, sortable, readable, discoverable, and self-described.

The main test for constructing name rule should be in questions:

· In what kind of places can I see this name?

· How easily can I work with the name in these places?

“Work” means to read, to find, and to understand sense without errors. Usually we work with names in lists. Think about these places like about namespaces in programming languages.

Short names:

Create as short as possible name.
Use abbreviations only in restricted cases. See article the “Abbreviations”http://msdn.microsoft.com/en-us/library/141e06ef(VS.71).aspx
Use prefixes and suffixes only to differentiate names.
If you see one word in several places of the full name, consider this as a bad signal. Try to redesign the terms used in the names.

Sortable:

Create “sortable” name. That means using more generic/important part of the name in the leftmost position. For example, prefer the name Folder_20090515 than the name Folder_05_15_2009.

Readable:

See http://en.wikipedia.org/wiki/Typography

Use the name case compatible with the well-known practices in programming languages/protocols with respect to upper and lower cases. For instance, the XML namespaces (URL) should be in lower case format, but the other names should be in the Pascal format. See the “Capitalization Styles” article [http://msdn.microsoft.com/en-us/library/x2dbyw72(VS.71).aspx ]
If the word with specific case is widely used in company, don’t force to change it to the Pascal format.
Decorate infixes, prefixes or suffixes with lower case and with undrscore. For example: TicketBatch_type, Source1 _and_Source2_to_Target, msg_MyRequest.

Discoverable:

The name should be discoverable. That means we easily should understand by the name of the artifact where we can find the artifacts or additional information about it. Names should link artifacts. Say, the schemas with XML namespace the http://domain.company.com/solution/project/2009-05-14 definitely should be in the project the Company.Domain.Solution.Project and in assembly the Company.Domain.Solution.Project.dll.

Self-described (Semantics):

Create name form the “business point of view” not form the “developer point of view”, especially the name exposed outside solution.
Don’t use generic terms in the names. Examples: Send, Receive, Service, Message, Transformation, Schema, Map, Orchestration, BizTalk.
Place a frequently used term into the shared dictionary with comment about where do use it and do not use.

Full, Short, and Composited names:

Several artifacts have the full names (composited names) and short names. Usually we can easily understand what name we use in the specific context.

· If we need to add one or more logical grouping into the name, use the composited names created with words separated with dot. In specific contexts the separation symbol can be different, like underscore, dash or backslash. Each <…ShortName> can be a composite name. Composite names can be used for each part of the full names, as Company, Domain, Solution, and Project. For example, you can use name the Schemas.Niem for the project short name.

3 General names

3.1 Syntax

3.2 Examples

· Solution name: short name – MySolution, full name - MyCompany.MyDomain.MySolution

· Project name: short composite name – Schemas.Niem; full name - MyCompany.MyDomain.MySolution.Schemas.Niem

4 Names inside BizTalk Solutions

4.1 Solutions, BizTalk applications

We use term the solution in the Visual Studio meaning, it is a name of the solution we see in the Solution Explorer window in the Visual Studio. Sometimes we mix solution and project terms but not in this document.

BizTalk application is named as solution.

For example, for the MySolution solution use the MyCompany.MyDomain.MySolution full name.

4.2 Projects, Assemblies, .NET namespaces

We use term the project in the Visual Studio meaning, it is a name of the project we see in the Solution Explorer window in the Visual Studio.

For example, for the MyProject project use the MyCompany.MyDomain.MySolution.MyProject project full name, the MyCompany.MyDomain.MySolution.MyProject assemblyname, and the MyCompany.MyDomain.MySolution.MyProject namespace.

We should separate the .NET namespaces and the XML namespaces. They are different things and used in different places. For brevity we use term the namespace exactly for the .NET namespace and always use the term the XML namespaces for the XML namespaces.

Note: After creating a new project, copy the project name property to the project properties the “Assembly Name” and the “Default Namespace”.

4.3 Folders

Use the predefined folder structure on all development machines and, if it is possible, on the Test and Product servers. This eliminates the errors and deployment effort because several BizTalk artifacts use the absolute file paths in configuration.

<SolutionsRootFolder> =: c:\Solutions

For example, the MyCompany.MyDomain.MySolution.MyProject project is placed in the c:\Solutions\MyCompany.MyDomain.MySolution\MyProject folder.

5 XML namespaces

Use very strict rules for the XML namespaces, because the XML documents expose the data interfaces to the outer world. The XML namespaces should follow the industry standards and the corporate standards. The exposed interfaces are immutable, why we should use versioning for XML namespaces. XML namespaces work as global unique identifiers for nodes of the XML documents.

We must use the URL or URN formats to XML namespaces. Feel free to use one of these standards. See the “Namespaces in XML 1.0 (Second Edition)”http://www.w3.org/TR/xml-names/ for more information.

[TBD: Here I use URL format because it is widespread and users are more familiar with it. Note two confusing things about using URL as an XML namespace. First, the URLs are used as addresses. URL in XML namespace works mostly as a global unique identifier, not as an address. Second, reverse names order in URLs, for example, not the http://com.company.domain but http://domain.company.com .]

[TBD: Here I use the URL reverse order for the XML namespaces. Not for the full name but for the “before-solution” part of the name, for the first part of the name before first backslash. If you don’t want to use XML namespaces that works as the web addresses, consider to use the XML namespaces in the generic, nonreversible, sortable order.]

Compose the URLs for the XML namespaces in the reversal order like http://domain.company.com/solution/schemas/2009-05-15 and add the “com” part of URL.

<TargetNamespace> =:
http://<Domain>.<Company>.com/<SolutionShortName>/<ProjectShortName>/<Version

<Version> =:
<date>
[in YYYY-MM-DD format]

[TBD We considered using two version formats. One format is like 1.0.0.1, the format used for the .NET assemblies. See http://msdn.microsoft.com/en-us/library/51ket42z.aspx . The second version format uses a date. Use the first one only if you can implement strict versioning rules, and in this case you can create/have some version approval procedure. We use here the date format.]

5.1 Additional Rules for XML namespaces

Use the same XML namespace for all schemas in one project. Schemas in one project (with equal XML namespace) are differentiated by the root node names. Do not place the root node name inside the XML namespace.
Use the project creating date for the first versions of all schemas inside project.
Use the current date for the second and next versions.
Create the new version only if the old one is published to production (test) environment. Do not create new versions inside development cycles.
Use the YYYY-MM-DD date format to make the names “sortable”. Do not use MM/DD/YY format.
URLs are case-sensitive (Yes, they are!) but do not use the upper case letters in XML namespaces.

For example, for the MyCompany.MyDomain.MySolution solution and the MyProject project the XML namespace should be the http://mydomain.mycompany.com/mysolution/myproject/2009-05-15 for the first versions of all schemas, if this project was created in 2009-05-15.

6 BizTalk artifacts

6.1 Orchestrations, Schemas, Pipelines

Names of these artifacts appear together with BizTalk application name mostly everywhere. We don’t need to use composite names. Use simple names for the names.

Note: If you see one word in several places of the full name, consider this as a bad signal. Try to rethink the terms used in the names. I repeat this rule here, because exactly in the full names of orchestrations, schemas and pipelines you can frequently see the repetitive words.

6.2 Maps

<Map> =:
        <SourceSchema>_to_<DestinationSchema> [for one-to-one map]
       <SourceSchema1>_and_<SourceSchema2>_to_<DestinationSchema> [for two-to-one map]
           …

If it is possible do not change the schema names in the map name. If the map name is excessive long, cut the schema names, but use the same cut rule for all map names.

6.3 Ports

Ports are the primary artifacts of the BizTalk solution. But in contrast to orchestrations they are used through the BizTalk application boundaries in many places, that is why, we have to use the composite names for the ports like for assemblies.

Do we need to separate one and two-way ports and send and receive ports, for example be “R_” or “SR_” prefixes? Do they mix up in lookups or in lists? No. And answer to the first question is “No”, do not use prefixes in the port names.

Do we have to use the transport/protocol qualifiers in the port names, like .FILE or .SOAP? No. One port could use several protocols. Moreover port is on the upper level architecture than transport. But for dynamic ports the transport name can be the main part of the port name.

Do we have to use message type in the port names, like .Request? No. One port could work with several message types. But frequently the message type can be used for the port name.

Try to understand the main purpose of the port and use it in the name. For example, link the port name with transport for dynamic port; or link the port name with the partner name, or with message type.

For example, MyCompany.MyDomain.MySolution.MyPartner.

7 Orchestration Naming Conventions

7.1 Orchestration artifacts

Use the prefixes to differentiate the artifacts in the XLang expressions. These artifacts are not usual .NET objects. They are used in different language context and sometime they use different language syntax. Prefixes really help to work with these artifacts.

<RoleLink> =:
roleLink_ + <Name>

7.2 Orchestration artifact types

We can use one suffix the “_type” for all different types because different types are seen only in the different lists and never mixed. For instance, we can never see the port types together with message types.

<ArtifactType> =:
<ArtifactName> + “_type”

7.3 Port Shapes

where

	Send port	Receive port	Send-Receive (Solicit-Response) port	Receive-Send (Request- Response) port
prefix	S_+	R_+	SR_+	RS_+

For example, S_ OrderAck.

Notes:

The Port shapes are the real names, the names of the .NET objects. We can’t use spaces inside.
In the Orchestration view there are generic lists the “Ports” and the “Port types” that’s why we have to distinguish the ports with different Communication directions and pattern.

7.4 Orchestration Workflow Shapes

Problems with orchestration shapes:

· Shapes are too small to display long names (only 12-18 characters).

· We have to “hover mouse over” shape or click shape to show Properties window to "understand" this shape, to understand what message it is processed.

Useful features:

· Feel free to use the same names for different shapes and use spaces inside the shape names. Shape names are not the “real programming names”. In reality they are the descriptions (excluding the Port shapes names); they are used only for description and for nothing more.

· Icons on shapes give us the useful information. Do not repeat the “icon information” by words. For example, if we change a name of Construction shape from “Construct Input message” to “Input message” we get more clear definition because we have the Construct icon + name.

· Shape names are used only in Orchestration Editor (excluding the Port shapes names). We don’t have to force any rules to make the “well-sorted” names (it's the main purpose of the prefixes).

· Use a Group shape to add description to a group of related workflow shapes. Group shape will display as much text as you want. Group shapes add a lot of documentation value to the orchestration.

7.4.1 Rules for shapes

Purpose of the orchestration and the most of the shapes is in processing the messages. We can unambiguously describe the messages by the message type. That is why in the most cases using the message type names gives us the main information about this message. That is why in the most cases using the message type names as the shape names gives us the main information about this shape, about message flow, about whole orchestration processing. Send shape with name "OrderAck" means ... exactly!

· Whenever it is possible use the MessageType of the processed message as a shape name.

· Do not repeat the type of shape icon by word.

· Do not repeat words from external shape name into the internal, nested shape name.

· Feel free to use spaces inside the shape names.

· Feel free to repeat the shape names.

7.4.2 Rules for specific shapes

Construct, Receive, and Send:

= name of the processed message without “msg_” prefix.

For example, [OrderAck]

Note: it’s easy to set and maintain this name: just copy part of it from Properties/Messages Constructed to Properties/Name. For example, from “msg_OrderAck” copy “OrderAck”

Transform:

= “from “ + name of the Source message

For example, [from OrderAck]

Note: it’s easy to set and maintain this name: just copy it (or part of it) from Properties/Input Messages to Properties/Name. For example: From “msg_OrderAck” copy “OrderAck”

Assignment:

No strict rules, only advice:

Name it like the methods in classes. But cut a verb if it possible. Use “set” and “get” if it possible.

For example, [set OrderAck]

8 BizTalk artifact and project places

Consider this chapter as “out-of-scope”. I placed it here because we discussed the folder names here. The files placement is a separate and wealthy topic. Here are only main considerations.

· If the project is simple, place all files in one project.

· If we want to use some files for references from other projects, place these files in separate project.

· Place artifacts to different projects if these artifacts have different refactoring lifecycle. For example, the Niem standard schemas are never changed then place them to the separate project. The maps are changed more frequently than schemas and we could place schemas and maps to the separate projects.

· Don’t place the technology-specific schemas and maps away from the orchestration they used for. For example, for the SQL port we generate a (technology) schema and usually create the map to transform the original schema to this (technology) schema. Place these schema and map together with orchestration, not into the Schemas/Maps projects.

8.1 Project subfolders

In the BizTalk project folder add the subfolder the Tests. Use it to the unit tests. Inside the Tests create subfolders: In, Out, TestMessages.

For the BizTalk project with different artifact files you can add the solution folders the Schemas, Orchestrations, and Maps in case you have several schemas, orchestrations, and maps there.

9 Out of scope

Several BizTalk artifacts are out of scope this naming convention:

· BRE artifacts: Rule sets, Vocabularies, etc.

· BAM artifacts: Activities, Views, BAM Definitions, Tracking Profiles

· Parties, Role links

· Itineraries from the ESB Toolkit 2.0. I think the ESB Toolkit is a part of BizTalk 2009, because it is delivered in binary format, supported by Microsoft, has a lot of tooling, including Itinerary Designer.

See also

1. “Naming guidelines for the .NET Framework types” in MSDN [ http://msdn.microsoft.com/en-us/library/xzf533w0(VS.71).aspx ]

2. “Naming Convention” in Wikipedia [ http://en.wikipedia.org/wiki/Identifier_naming_convention ]

3. “Abbreviations” in MSDN [http://msdn.microsoft.com/en-us/library/141e06ef(VS.71).aspx ]

4. “Readability” in Wikipedia [See http://en.wikipedia.org/wiki/Typography ]

5. “Capitalization Styles” in MSDN [http://msdn.microsoft.com/en-us/library/x2dbyw72(VS.71).aspx ]

6. “Assembly Versioning” in MSDN [ http://msdn.microsoft.com/en-us/library/51ket42z.aspx ]

7. “Namespaces in XML 1.0” [ http://www.w3.org/TR/xml-names/ ]

8. URN in Wikipedia [ http://en.wikipedia.org/wiki/Uniform_Resource_Name ].

9. URL in Wikipedia [ http://en.wikipedia.org/wiki/URL ].

Appendix

Syntax

<Word> =:
[A-Za-z1-0]*

<CompositeWord> =:
<Word>.<Word>[.<Word>]

<{Any}ShortName> =: <ShortName>
[replace {Any} with any term. For example, term Solution creates a <SolutionShortName> term]

<SolutionsRootFolder> =: c:\Solutions

<Version> =:
<date>
[in YYYY-MM-DD format]

<Map> =:
<SourceSchemaShortName>_to_<DestinationSchemaShortName> [for one-to-one map]
<SourceSchemaShortName1>_and_<SourceSchemaShortName2>_to_<DestinationSchemaShortName> [for two-to-one map]
…

Instructions

Keep the names of some BizTalk artifacts in the lists, a list per solution. Create, at last, a list of the XML namespaces of all schemas. Keep them in one list; it forces developers to use naming convention, because inconsistencies in the names are visible inside these lists on the development stage.

1. Create a new name in the list below.

2. Copy it to the name property of the new BizTalk artifact.

XML namespace List

Port List

BizTalk: What features we would like to see in future releases

Leonid Ganeline — Wed, 01 Apr 2009 21:08:44 GMT

BizTalk: What features we would like to see in future releases

After the MVP Summit [http://geekswithblogs.net/LeonidGaneline/archive/2009/03/09/microsoft-global-summit-mvp-most-valuable-professional-2009-pictures.aspx] we, the BizTalk MVPs, have been asked to give feedback to the product team on what features we’d like to see in future releases.

I created the list of features and separated it into parts. One is from the global, crazy things than never be implemented. But why don’t imagine? Second one is from the small things.

Let start from the big, crazy things:

More flexibility in the deployment and licensing.

What I mean:

1) Why don't move the BizTalk as a product to cheap segment? (cheap? No, let's talk about bigger)

a) BizTalk + SQL Express on one box. Then creating a farm of the BizTalk applications would be easier. Load Balancing is moved outside, before the farm. Maybe create a separate product the "BizTalk load balancer"?

b) Moreover, ship the preconfigured VHD with BizTalk + SQL Express + Windows Server. With possibility to create the farm from these VHDs. (Not so easy because now we have strict limit to the SQL and BizTalk server names, and that limit should be fixed anyway.)

2) Why don't move to the expensive segment?

a) Shift to the Windows HPC? What about this? One time the BizTalk was pioneered as the first enterprise level product by Microsoft, created on .NET. Using the HPC methods to get rid of SQL completely, perform all persistency in memory to get super-low latency product.

3) Reiterate the architecture with all spectrum new Microsoft technologies. Is it possible to extract the MessageEngine, the OrchestrationEngine, and the UnifiedInputOutput Engine to separate products?

4) At last extract the BRE to separate product or free library! Now it is dead. Give BRE a shock treatment.

5) Redesign the error processing for different stages of the message processing. Now there are different approaches are used in many places or nothing at all.

6) Extract the LOB Adapters to separate product on the “buy if you need it” basis or to the free library. Create the Internet store and sell adapters for small money to everyone not only to the buyers of the BizTalk. Sell them not in packages but per adapter.

Now the small things:

Many years there were no improvements in very basic things, the things the developers are using in everyday tasks. I mean: adapters: File, SQL; pipelines; schema and map editors; additional shapes in Orchestrations, etc.

1. Adapters:

a. File adapter (for example): add the possibility to use the file names to order delivery (process files in the sorted order, many variations J ); implement the file receive filters with regEx on the file names; add possibility to copy files to other folder after consuming (to make some sort audit/resubmit storage, in forums this question is repeated and repeated, "How we could make a copy of the original files?")

b. SQL adapter: OMG, there should be many changes in real-time functions and much more improvements in the design-time appearance. I'd prefer to use not the standard Wizard to generate SQL port artifacts but some sort rich-UI window to set up different stages of artifact creation and the real-time parameters.
I am sure the brainstorm by BizTalk MVP should generate a lot of good ideas.

c. Additional adapters: like Task Scheduler; Nope; Fan-out/Garbage (to consume messages without subscribers in the pub/sub application); PowerShell (for additional processing of the input/output messages). Brainstorm this! Buy these adapters from the third-parts, unify them and include to the standard package.

d. Please, make InfoPath integration with BizTalk easy as hell!!!! (Isn't it a prime purpose of InfoPath??? To integrate the BizTalk with human?)

2. Pipelines (pipeline components): where are? the xml <-> PDF ; zip/unzip ; map (yeah! what is wrong with it?); different coders/decoders; symbol (regex) replacement (for EDI processing it MUST be!); PowerShell (for additional processing of the input/output messages); "create message" pipeline; Excel/Word ; Html ; simple EDI (to very basic EDI processing); promote properties from the text; etc. . Brainstorm this!

3. Add the error processing to the standard interfaces of the custom pipeline components.

4. Transmitting raw/binary data. Frequently we only use the BizTalk to transfer and route the documents (messages) without transformation. Now it is the second-class citizen scenario.

5. Orchestrations: additional shapes as: CreateMessage !!!; throwEvent; Pipeline transformation (?).
Make shapes resizable, otherwise we should use odd methods to make the Orchestration more descriptive. How we could show Orchestration to business people if we cannot add enough text to shapes???? Add color to shapes.
Make the Expression Editor window resizable, dockable, fully InteliSensed, etc. !!!!!!!!!!!!!!!!!!

6. Schema editor: Use the schema node icons for more useful information, definitely, for cardinality (say, different colors for "max occurs = *", for "min occurs = 0"). Maybe add separate window for the referenced (included) schemas?

7. Map editor: add functoids [IfElse]... Make improvements to the Script functoid!!! Separate to InlineScript; ExternalScript (ExternalMethod?); XsltScript...; make InteliSense to the Inline Script and Xslt script windows. Use the source/target node icons to more usefull information. See http://geekswithblogs.net/LeonidGaneline/archive/2005/12/23/64004.aspx - it is an old list but still actual.

8. Ports: Please, make ports the first-class artifact as an Orch. Why we should use the orchestration assembly just to store the port type??? Why do we use the "logical port" term in Orchestration editor? It is not a port, it is just an endpoint. Orchestration has the endpoints and the Port has the endpoints. My perception it should be big redesign of the "port part" of the BizTalk architecture. Right now it is a mix of adapters, pipelines, maps, etc. without strict regulation, strict design. This mix is very static, could not be used in structured way to improve the development.

The question to us was: 1) What was/is THE GOOD stuff about the area? 2) What is just plain BAD? 3) What is UGLY about the feature area e.g. if you were developing this feature what would you do to improve it.

I am sorry but I used the same words. To me “UGLY” and “BAD” are just synonyms for ”rudimentary implemented” and “implemented in beta version”.

1) Monitoring, operations. Now it is UGLY. No real real-time :). See how it works on HPC Windows, for example.

2) Debugging is plain BAD.

3) BRE is good, but EVERYTHING for development BR, everything is UGLY. Development, testing-debugging (come on, have somebody seen the test text window in Composer? the text inside it????), deploying. BRE now is good, very good to sell the BizTalk to business people, and BRE is just pain in ass for developers. Examples?
http://geekswithblogs.net/LeonidGaneline/archive/2006/12/27/102156.aspx
http://geekswithblogs.net/LeonidGaneline/archive/2006/10/16/94245.aspx
Now as a rule of thumb, I use the BRE for POC projects and never for real projects.

4) To me BAM is kind of partly implemented. Run-time is very good. Design-time is BAD.

5) ESB. IMHO if the BizTalk gets the Web-service (or metadata) repository, it has ALL ESB functionality and could be named ESB to business people. MVPs discussed this many times on the Summit.

6) Of course, low latency must be implemented.

BizTalk: Debugging: Why is it so difficult?

Leonid Ganeline — Thu, 09 Oct 2008 21:12:08 GMT

Sometimes in the BizTalk forums such questions appear.

"...I have been using event log to test variables. I was hoping there was a way to watch it process in VS like you can debug a forms app."

There is a difference in the usual app and in the "BizTalk app".

When you start usual app it is exactly this app and you could debug it. When you "start" the BizTalk app that could means *many* *instances* of this BizTalk app could works simultaneously.

With BizTalk you have an intermediate layer, the host. This host (actually it is a host instance but it doesn't matter for this case) manages the app *instances*. It could start many app instances simultaneously and then dehydrate, rehydrate, stop, abort them without our intervention.

Ideally we need to attach a debugger to exactly one app instance (which one?) or create a debugger working with many app instances simultaneously.

It is one of the source of the complexity of debugging the BizTalk apps.

BTW What is a BizTalk application at all?

Please, let me know, what do you think.

BizTalk: Patterns: discussion "BizTalk, From Hub/Spoke to ESB"

Leonid Ganeline — Wed, 03 Sep 2008 05:13:42 GMT

There is an interesting discussion "BizTalk, From Hub/Spoke to ESB" in the MSDN BizTalk forum: [http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=3478379&SiteID=1]

WCF: Main Types: Outline

Leonid Ganeline — Tue, 24 Jun 2008 22:01:28 GMT

Notes

· Bindings are spread across several namespaces.

· DataContractAttribute is placed in the System.Runtime.Serialization namespace not in the System.ServiceModel namespace.

· Configuration sections are spread across several namespaces.

· ClientSection and ServicesSection: First has the structure client/endpoint, the second – services/service/endpoint. Asymmetric.

System.ServiceModel.dll

System.ServiceModel

· ServiceHost

· ChannelFactory<TChannel>

· ClientBase<TChannel>

Contracts:

· ServiceContractAttribute

· OperationContractAttribute

· MessageContractAttribute

· FaultContractAttribute

Bindings:

· BasicHttpBinding

· NetMsmqBinding

· NetNamedPipeBinding

· NetPeerTcpBinding

· NetTcpBinding

· WS2007FederationHttpBinding

· WS2007HttpBinding

· WSDualHttpBinding

· WSFederationHttpBinding

· WSHttpBinding

Exceptions:

· CommunicationException

· …

System.ServiceModel.Activation

· ServiceHostFactory

· WorkflowServiceHostFactory

System.ServiceModel.Activation.Configuration

· DiagnosticSection

System.ServiceModel.Channels

Interfaces:

· IChannel

· IChannelFactory<TChannel>

· IChannelListener<TChannel>

Abstract classes:

· Binding

· BindingContext

· BindingElement

· ChannelFactoryBase<TChannel>

· ChannelListenerBase<TChannel>

· CommunicationObject

· Message

· MessageHeader

· …

Bindings:

· CustomBinding

Binding Elements:

· TransportBindingElement

· MessageEncodingBindingElement

· …

System.ServiceModel.Configuration

Sections:

· BehaviorsSection

· BindingsSection

· ClientSection

· DiagnosticSection

· ServicesSection

Element: System.Configuration.ConfigurationElement

· BaseAddressElement

· …

ElementCollection: System.ServiceModel.Configuration.ServiceModelEnhancedConfigurationElementCollection<ComMethodElement>

· BaseAddressElementCollection

· …

System.ServiceModel.Description

· ServiceEndpoint

Behavior interfaces:

· IContractBehavior

· IEndpointBehavior

· IOperationBehavior

· IServiceBehavior

Descriptions:

· ContractDescription

· FaultDescription

· ServiceDescription

System.ServiceModel.Dispatcher

· ChannelDispatcher

* ClientOperation

* ClientRuntime

· EndpointDispatcher

Filters:

· ActionMessageFilter

· MatchAllMessageFilter

· MessageFilter

· …

System.ServiceModel.MsmqIntegration

· MsmqIntegrationBinding

System.ServiceModel.Security

Credentials:

Encoders:

System.ServiceModel.Security.Tokens

Tokens:

SecurityTokenParameters:

System.Runtime.Serialization.dll

System.Runtime.Serialization

Attributes:

· CollectionDataContractAttribute

· DataContractAttribute

· DataMemberAttribute

· EnumMemberAttribute

System.WorkflowServices.dll (v3.5)

System.ServiceModel

· WorkflowServiceHost

Bindings:

· BasicHttpContextBinding

· NetTcpContextBinding

· WSHttpContextBinding

BizTalk: Architectural Tips: Custom Message Audit

Leonid Ganeline — Wed, 30 Apr 2008 03:14:16 GMT

The message audit can be implemented by BizTalk Tracking (BizTalkDTADb) database.

But there is one big "But". This database has predefined, undescribed structure, it should be used only by special BizTalk tools as HAT. We cannot change the audit points, cannot freely create user queries, cannot change structure of this database.

It is not so hard to create the custom audit, it is the standard practice to make the custom audit database and gather data for it from the BizTalk application with additions for the Orchestrations.

It is good practice to use some standard for the audit architecture. Here I give you two of them.
They are pretty easy and repeat the "Generic Error handling" pattern. They use the Direct port subscription.

I. One audit send port, the audit messages are created in different ways. All audit messages have the same message type. The Audit send port is subscribed to the one audit message type.

II. One audit send port with set of the maps, the audit messages are created by these maps.The Audit send port group is subscribed to the all audited message type.

We create the standard audit message type; send all audit messages in all audit points to the Direct send port shape; create the Direct send port with subscription to this audited message type. That is all.

[The audit points are the all audited milestones as receiving/sending the message; transforming the message; routing the message , etc.]

ToDo list:

Create the generic_audit schema. It can, for example, include elements: "Operation datetime", "Operation type"(Send/Receive/...), "Original message parameter (like File Name)", "Success/Fail status", "My_Specific_parameter" etc.
Create simple maps to produce the generic_audit schema from all other messages. It is not necessarily all audit parameters be mapped from the source message. Some of them might be produced inside map or expressions, for example, the "Operation datetime" parameter.
Create the Audit Send Direct port with these maps in the entry. Make filters for all audited message types, i.e. subscribe to all these message types.
If we want to save the audited messages in the database: Create the Audit database. The simplest database might be with one table and one stored procedure to fill-in this table.

FYI You can see in MSDN forum (http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=2452186&SiteID=1) a comment of Kent Weare [BizTalk MVP] about using BAM for auditing:

"...On at least a couple of occasions I have heard MS employees endorse BAM for auditing. We brought a Microsoft consultant to review our architecture and design with respect to auditing. He suggested we use BAM for this purpose. Note he did not endorse message body tracking inside of BAM. The other place that I have seen it endorsed is at TechEd at one of Darren Jefford's sessions.
Without a doubt building up the required BAM infrastructure is not a trivial task(especially in an enterprise scenario). Keeping things simple is often the best solution, so I do think that you have brought up some valid points. ..."

One more discussion about using HAT for tracking messages is here (http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=1991237&SiteID=1).

BizTalk: Debugging the BizTalk applications with the direct test send ports

Leonid Ganeline — Sun, 27 Apr 2008 22:54:27 GMT

" ...I have published an orchestration as a web service and consumed it from an aspx page. now, how do I see if I triggered the orchestration and how and where do I get the output (transformed message of the orchestration)? ..."

It is a very common question, how to debug the BizTalk applications.

Have you ever tried to use the BizTalk Orchestration Debugger? So painful experience. Never see the BizTalk developer who is using it in real work.

One simple decision is creating the direct test send ports. (http://blogs.msdn.com/kevin_lam/archive/2006/07/07/659214.aspx, http://geekswithblogs.net/LeonidGaneline/archive/2006/12/18/101541.aspx)

Test port is a port with the filter expression (http://msdn2.microsoft.com/en-us/library/aa578042.aspx, http://msdn2.microsoft.com/en-us/library/aa547903.aspx). This expression creates the subscription to the message you interested in.

All sent and received messages go through the MessageBox. When they pass MessageBox you can create additional subscription to these messages and intercept copies of these messages.

The subscription can be to the message type or to the publisher parameters like Receive port name.

This technique has great benefits:

The direct test send ports do not disturb the original message flow (excluding the rare case when all main subscriptions to the debugged message have got unenlisted).
They can be easily switch on/off.

As a matter of fact I use two methods on the regular basis. (The second one is the old good Trace.WriteLine(..) with DebugView.exe as a receiver of the debug output.) That means I always create the orchestrations with the Expression shapes (for Trace.WriteLine)) after all message receives/transformations, in the begin/end all branches, plus I create the test send ports for almost ALL messages.

Yes, for ALL output messages.

It is easy and fast to create the test send ports in the binding file without using UI.

And I always deploy these test ports to the production. It saved me a lot of time in deployment and maintain.

BizTalk: Contract-Driven Messaging

Leonid Ganeline — Fri, 04 Apr 2008 03:25:16 GMT

I was stopped by the words "Contract-Driven Messaging" in the article "An XML Guru's Guide to BizTalk Server 2004, Part I" by Aaron Skonnard ("Contract-Driven Messaging"
http://msdn.microsoft.com/msdnmag/issues/05/11/ServiceStation/#S6 )

What does in mean the "Contract-driven" in the BizTalk?
The contract in the BizTalk means the same thing as in WCF, the signatures of the objects (documents, messages). But not only the the objects with entry data but the with context data. For example, the addresses of the endpoint where those documents came, the delivery transport protocol etc.

What is it means from the BizTalk point of view?
In BizTalk almost all messages are in the internal XML format and have a type (which compounded of the target namespace and the root element of the schema of this message). Contract-Driven Messaging from the BizTalk point of view is the Message-type driven messaging, because in the BizTalk the "Contract" is exactly the message type. Plus the context of the message!

How has the message got the type?

The type is defined explicitly if the message comes to the BizTalk in the XML format.
How has the message got the type if it comes in other format, say comma-delimited text format? In this case the BizTalk uses the content and the context information to produce the message type. For instance, in EDI document the message type produced by the several data fields inside document and, maybe, the context information as an FTP address of the receive location of this document. And we have to explicitly map those input data set to the message type (as a namespace + the root node). Usually we don't have the values from the input data to pass it implicitly or by some rule to the message type.

OK, and what does it mean the messaging "driven" by?
It means transformation and routing.
Thansformations like mapping, batching/debatching, aggregating, copying.
Routing is a choosing the subscribers of the message.

And one more interesting aspect of the "contract-driveness". With it the data could be processed automatically, without human intervention. Data has all intrinsic information to be processed.

BizTalk: Accumulating messages in MessageBox, Lifespan of the messages

Leonid Ganeline — Wed, 14 Nov 2007 21:15:09 GMT

I hit this problem when were creating the orchestration to periodically poll the outer service. The orchestration was with loop inside.

The problem was I created the message inside this loop. And the instance of this message were not disposed after each cycle. The BizTalk does not have such functionality as explicit deleting the messages inside orchestration.

The orchestration was created for running long time without restart. That means that this orchestration was collecting enormous amount those indisposed messages. And all those messages were inside the MessageBox, of course. It was wrong!

After long and very interesting investigation the issue was resolved.

The source of the problem was in the transactional scope where I created the indisposable message. This scope was inside the loop. The message was recreated on each cycle of the loop.

It is interesting but a lifespan of the message recreated inside the transactional scope included in the loop and the message recreated outside the transactional scope in the loop is different. The first one is disposed only after closing the orchestration instance. The second one is disposed after last using. (I think so but could not find a hint in the BizTalk information sea.)

After discovering this the fix was simple. I moved the transaction scope outside of the loop.

BizTalk - Data-driven application host server (?)

Leonid Ganeline — Tue, 13 Nov 2007 17:06:45 GMT

Now the BizTalk is recognized and advertized mostly as an Integration server.

There is an architectural pattern for using BizTalk. It is used:
* as an integration engine, as an integration layer between applications (usually and as a data transformation layer);
* for the workflow applications (including the long-running workflows);
* as a host for Business Rule Engine applications.

I think the BizTalk Server is more then that.

Let's think of BizTalk Server as a generic Application Server.
Now we have the Windows Server OS as an Application Server. We have the IIS as the Application Server.
Windows can host the standalone applications and Windows service applications. We can start the applications by hand, schedule its start by the Scheduled Task utility, setup the special developed applications as the Windows Services.
IIS hosts the Web applications.

Let's look at the BizTalk Server as a generic Application Server. Application Server for our usual standalone applications.

The .NET applications can easily start from the Expression shape of the simple Orchestration. There is little redundancy in development.

What the advantages we have using the BizTalk as an Application Server for standalone applications?

Data-driven applications:
Somewhere in the system the data appears. It can be a new file copied in a folder, new file received by FTP, new data in SQL database, new messages in the message queue, new data in the SAP/R3 application, new e-mail. Those data triggered the BizTalk application.
Data is appeared and this starts the appropriate application.
Say a hundred new files appears in the folder (or a hundred rows appeared in the SQL database). And this starts a hundred appropriated applications to concurrently process this data.
This is not easy to implement this data-driven application management in the usual standalone application.
BizTalk implements many transport and application adapters to transfer data to/from BizTalk applications, like File, FTP, MSMQ, SAP/R3, JD Edwards, etc.

Long-running applications; data correlations:
Say our application sent data to the remote partner then after several minutes or days the partner modified data and sent it back. How can we facilitate the application to wait this response? How can the application survive the server restarts? How can we facilitate the situation when thousand applications are waiting the responses and flooding the server memory? How can we correlate the response data with appropriate application?
BizTalk by design works all these cases.

Scaling up and scaling out:
Having BizTalk as an Application Server the developers can redesign the architecture from big, monolithic application to the small, isolated business-unit application set. BizTalk helps to transfer data between those small applications, helps to correlate data. This ways to high scalable application. Moreover the BizTalk has the intrinsic functionality to manage the high availability and high performance requirements by the special host mechanism.

Yes, it's true that the data-driven applications get the well-organized data which definitely is the output from other applications. Is that mean we've returned to the point where we have gone because it means the old paradigm, the integration level where the BizTalk always was positioned?
The integration tool is the intermediate level tool. It is not the self-independent tool.
But seems the BizTalk is the self-independent Application Server for our usual standalone applications.

Any ideas?

Best regards,