BizTalk_Mapping

Conversion of BizTalk Projects to Use the New WCF-SAP Adaptor

Geordie — Sun, 06 Feb 2011 18:45:03 GMT

We are in the process of upgrading our BizTalk Environment from BizTalk 2006 R2 to BizTalk 2010. The SAP adaptor in BizTalk 2010 is an all new and more powerful WCF-SAP adaptor. When my colleagues tested out the new adaptor they discovered that the format of the data extracted from SAP was not identical to the old adaptor. This is not a big deal if the structure of the messages from SAP is simple. In this case we were receiving the delivery and invoice iDocs. Both these structures are complex especially the delivery document. Over the past few years I have tweaked the delivery mapping to remove bugs from original mapping. The idea of redoing these maps did not appeal and due to the current work load was not even an option. I opted for a rather crude alternative of pulling in the iDoc in the new typed format and then adding a static map at the start of the orchestration to convert the data to the old schema.

Note WCF-SAP data formats (on the binding tab of the configuration dialog box is the ‘RecieiveIdocFormat’ field):

Typed: Returns a XML document with the hierarchy represented in XML and all fields being represented by XML tags.
RFC: Returns an XML document with the hierarchy represented in XML but the iDoc lines in flat file format.
String: This returns the iDoc in a format that is closest to the original flat file format but is still wrapped with some top level XML tags. The files also contained some strange characters at the end of each line.

I started with the invoice document and it was quite straight forward to add the mapping but this is where my problems started. The orchestrations for these documents are dynamic and so require the identity of the partner to be able to correctly configure the orchestration. The partner identity is in the EDI_DC40 segment of the iDoc. In the old project the RECPRN node of the segment was promoted. The code to set a variable to the partner ID was now failing. After lot of head scratching I discovered the problem was due to the addition of Namespaces to the fields in the EDI_DC40 segment. To overcome this I needed to use an xPath query with a Namespace Manager. This had to be done in custom code.

I now tried to repeat the process with the delivery document. Unfortunately when we tried to get sample typed data from SAP an exception was thrown.

The adapter "WCF-SAP" raised an error message. Details "Microsoft.ServiceModel.Channels.Common.XmlReaderGenerationException: The segment or group definition E2EDKA1001 was not found in the IDoc metadata. The UniqueId of the IDoc type is: IDOCTYP/3/DESADV01/ZASNEXT1/640. For Receive operations, the SAP adapter does not support unreleased segments.

Our guess is that when the WCF-SAP adaptor tries to down load the data it retrieves a data schema from SAP. For some reason the schema does not match the data. This may be due to the version of SAP we are running or due to a customization. Either way resolving this problem did not look easy.

When doing some research on this problem I found an article showing me how to get the data from SAP using the WCF-SAP adaptor without any XML tags.

http://blogs.msdn.com/b/adapters/archive/2007/10/05/receiving-idocs-getting-the-raw-idoc-data.aspx

Reproduction of Mustansir blog:

Since the WCF based SAP Adapter is ... well, WCF based, all data flowing in and out of the adapter is encapsulated within a SOAP message. Which means there are those pesky xml tags all over the place. If you want to receive an Idoc from SAP, you can receive it in "Typed" format (in which case each column in each segment of the idoc appears within its own xml tag), or you can receive it in "String" format (in which case there are just 2 xml tags at the top, the raw xml data in string/flat file format, and the 2 closing xml tags).

In "String" format, an incoming idoc (for ORDERS05, containing 5 data records) would look like:

<ReceiveIdoc xmlns='http://Microsoft.LobServices.Sap/2007/03/Idoc/'><idocData>EDI_DC40 8000000000001064985620 E2EDK01005 800000000000106498500000100000001 E2EDK14 8000000000001064985000002000000020111000 E2EDK14 8000000000001064985000003000000020081000 E2EDK14 80000000000010649850000040000000200710 E2EDK14 80000000000010649850000050000000200600</idocData></ReceiveIdoc>

(I have trimmed part of the control record so that it fits cleanly here on one line).

Now, you're only interested in the IDOC data, and don't care much for the XML tags. It isn't that difficult to write your own pipeline component, or even some logic in the orchestration to remove the tags, right? Well, you don't need to write any extra code at all - the WCF Adapter can help you here!

During the configuration of your one-way Receive Location using WCF-Custom, navigate to the Messages tab. Under the section "Inbound BizTalk Messge Body", select the "Path" radio button, and:

(a) Enter the body path expression as: /*[local-name()='ReceiveIdoc']/*[local-name()='idocData']

(b) Choose "String" for the Node Encoding.

What we've done is, used an XPATH to pull out the value of the "idocData" node from the XML. Your Receive Location will now emit text containing only the idoc data.

You can at this point, for example, put the Flat File Pipeline component to convert the flat text into a different xml format based on some other schema you already have, and receive your version of the xml formatted message in your orchestration.

This was potentially a much easier solution than adding the static maps to the orchestrations and overcame the issue with ‘Typed’ delivery documents. Not quite so fast…

Note: When I followed Mustansir’s blog the characters at the end of each line disappeared.

After configuring the adaptor and passing the iDoc data into the original flat file receive pipelines I was receiving exceptions.

There was a failure executing the receive pipeline: "PAPINETPipelines.DeliveryFlatFileReceive, CustomerIntegration2.PAPINET.Pipelines, Version=1.0.0.0, Culture=neutral, PublicKeyToken=4ca3635fbf092bbb" Source: "Pipeline " Receive Port: "recSAP_Delivery" URI: "D:\CustomerIntegration2\SAP\Delivery\*.xml" Reason: An error occurred when parsing the incoming document: "Unexpected data found while looking for:

'Z2EDPZ7'

The current definition being parsed is E2EDP07GRP. The stream offset where the error occured is 8859. The line number where the error occured is 23. The column where the error occured is 0.".

Although the new flat file looked the same as the old one there was a differences. In the original file all lines in the document were exactly 1064 character long. In the new file all lines were truncated to the last alphanumeric character.

The final piece of the puzzle was to add a custom pipeline component to pad all the lines to 1064 characters. This component was added to the decode node of the custom delivery and invoice flat file disassembler pipelines.

Execute method of the custom pipeline component:

public IBaseMessage Execute(IPipelineContext pc, IBaseMessage inmsg)

{

//Convert Stream to a string

Stream s = null;

IBaseMessagePart bodyPart = inmsg.BodyPart;

// NOTE inmsg.BodyPart.Data is implemented only as a setter in the http adapter API and a

//getter and setter for the file adapter. Use GetOriginalDataStream to get data instead.

if (bodyPart != null)

s = bodyPart.GetOriginalDataStream();

string newMsg = string.Empty;

string strLine;

try

{

StreamReader sr = new StreamReader(s);

strLine = sr.ReadLine();

while (strLine != null)

{

//Execute padding code

if (strLine != null)

strLine = strLine.PadRight(1064, ' ') + "\r\n";

newMsg += strLine;

strLine = sr.ReadLine();

}

sr.Close();

}

catch (IOException ex)

{

throw new Exception("Error occured trying to pad the message to 1064 charactors");

}

//Convert back to stream and set to Data property

inmsg.BodyPart.Data = new MemoryStream(Encoding.UTF8.GetBytes(newMsg)); ;

//reset the position of the stream to zero

inmsg.BodyPart.Data.Position = 0;

return inmsg;

}

How to Map the Selection of the First Item in a Collection to the Final Document

Geordie — Wed, 13 May 2009 16:04:57 GMT

I recently came across a new mapping problem. I needed to make sure that the final document only had one address for a paper mill to satisfy the requirement of a standard documentation format (PAPINet). Usually the product is all made to order but every so often paper may be available from stock that was made in a different mill. The SAP system providing the information contains information on both mills. The challenge here was to remove the additional mills and only include the mill data for the first mill. In this case the mill information is not important as the ID for the paper (TAPI) includes a character that identifies the producing mill.

My first attempt was to limit the number of occurrences of the Mill element. BizTalk ignored the limit.

I then tried to use the 'Data Flattening' functoids with an 'Iteration' functoid. I set the 'Equal' functoid to receive the value from the 'Iteration' functiod and added a constant value of 1 (BizTalk must use 1 base in it's iterations). When the 'Iteration' functoid equals 1 it will resolve to true allowing the 'Data Flattening' functoids to only process the first occurrence of the Mill attribute.

The overall solution is a complex mapping from SAP to PAPINet. It uses a canonical structure to assist in the transformation. This problem was greatly simplified by using the canonical to canonical mapping (added to allow some complex calculations to be preformed).

How to block the creation of parent nodes when using the data flattening functoid

Geordie — Thu, 12 Mar 2009 20:17:27 GMT

I've had a few BizTalk maps that have caused me to stop and think as to how to get over the problem of suppressing nodes when the child node has no data. The issue arises if you use Data flattening shape to block the creation of an element. The child node is suppressed but the parent node is still created. I just found out how to block the parent node from being created so I’m posting it here so I can find it again the next time I need it.

By connecting the parent nodes to the functoid that provides the condition to the data flattening functoids the parent nodes will be suppressed.

Thanks to Saravana Kumar for posting the answer on his blog.

Geordie

Installing custom functoids

Geordie — Wed, 21 Jan 2009 22:46:51 GMT

I just got bit by this one a second time. So it is time to store the information here to avoid a third occurrence.

If you get an exception that the functoid was not found.

For example:- 'Exception Caught: Functoid not found:
guid({5DE500CC-45BC-454b-A23D-24449899042C})  funcid(6123)'

Check that the dll is installed in the mapper extension folder 'C:\Program Files\Microsoft BizTalk Server 2006\Developer Tools\Mapper Extensions' as well as the GAC.

Geordie

XTransformationFailureException: encountered while executing the transform…

Geordie — Wed, 03 Dec 2008 04:00:49 GMT

I recently encounter this error. It took a little bit of head scratching to work out why we were getting the error. The possible causes were quite high as we are using dynamically assigned maps (see code below).

Message Assignment Shape Code

SapOrderMapType = System.Type.GetType(SapOrdersMapName);

//transform(MyOutputMsg) = MyMapType(MyInputMsg);

transform (SapOrder) = SapOrderMapType(CanonicalPO);

System.Diagnostics.Trace.WriteLineIf(TraceEnabled, "Transform to SAP schema complete.", TraceCategory);

SapOrder(FILE.ReceivedFileName) = "PO_" + CustomerNumber + "_" + PONumber + "_";

System.Diagnostics.Trace.WriteLineIf(TraceEnabled, "SAP Order Msg Created", TraceCategory);

The error information identifies the transform as the problem but not what the problem is. The inner exception is no additional help- ‘Value cannot be null. Parameter name: extension Exception type: ArgumentNullException’.

The problem was eventually tracked down to the custom functoids. In the last map a new functoid was added to convert standard BizTalk dates into standard SAP dates. When the project was deployed the custom functoid dll was not updated.

Interestingly this error might occur intermittently as the functiod may not be needed every time the map is used.