BizTalk

Installing UDDI 3 Server on a BizTalk Dev Box

Geordie — Fri, 30 Dec 2011 06:47:18 GMT

I have just installed the UDDI 3 server on my BizTalk development environment. All looked good until I tried to open the publish page on the web interface. The page 'http://localhost/uddi/edit/frames.aspx' returned a page cannot be displayed error. The same error occured when I tried to open the Subscribe and Coordinate pages.

After playing around with the configuration for a while I tracked the problem down to the page using https. By connecting to the UDDI Service Console, right clicking on the UDDI in the left panel I was able to open the UDDI properties dialogue box. The require SSL for publication requests can be disabled on the Security tab.

P.S. Loading the UDDI Server with the Microsoft ESB Providers, tModels etc can be done using the "Microsoft.Practices.ESB.UDDIPublisher.exe" tool located at "C:\Program Files (x86)\Microsoft BizTalk ESB Toolkit 2.1\BinC:\Program Files (x86)\Microsoft BizTalk ESB Toolkit 2.1\Bin". In my case I used windows authentication to load the data into the UDDI service.

Geordie

Creating a Send Port that can Generate Multiple Flat File Formats

Geordie — Wed, 26 Oct 2011 05:46:19 GMT

I recently had a need to create a send port that could output a flat file but I could not determine which flat file schema to use at design time. Or at least I didn’t want to. I wanted to process multiple flat file formats with the same orchestration process and output with the same send port. I have my orchestration dynamically mapping the document to the appropriate flat file document schema. The generated orchestration message type is set to System.XmlDocument so the orchestration can handle the varying message schema types, but how to output them with a single port.

The problem is with the pipeline. The port uses a custom pipeline that uses a flat file assembler to generate the flat file. In the configuration of the assembler the flat file document schema to be generated is specified. So how to dynamically set the pipeline… I have yet to find an answer to this problem (I’ll let you know when I do!).

There is an easier solution to this problem. It is very simple once you know how the assembler and for that matter the receive pipeline disassemblers work. When we specify the schema in the components we are basically telling the pipeline which schema to use to process the message. So what happens if we do not specify a schema?

This is the interesting part, especially for the assembler component. BizTalk already knows which schema to use. Using the schema type (the combination of TargetNameSpace#RootNodeName) form the message, BizTalk can ask the configuration database which schema to use. Therefore by not specifying a schema in the flat file assembler you create a pipeline that can assemble any flat file BizTalk has a schema for. Although this works well for the XML disassembler as well it is not so easy for flat files. Like most the other pipeline components the disassembler can contain multiple disassemblers but unlike the rest the message flows through differently. As the message hits the first disassembler it tries to process the message, if it fails, it will try the next disassembler. When is succeeds in identifying the message and applying the schema it ignores the rest of the disassembler components and carries on down the pipeline. All other multiple pipeline components are processed sequentially. This information is displayed in the pipeline designer, but I never realised the significance.

Note the benefit of specifying the schema is for performance. There is more overhead if the pipeline has to search for the schema to use.

Hope this helps

Geordie

Conversion of BizTalk Projects to Use the New WCF-SAP Adaptor

Geordie — Sun, 06 Feb 2011 18:45:03 GMT

We are in the process of upgrading our BizTalk Environment from BizTalk 2006 R2 to BizTalk 2010. The SAP adaptor in BizTalk 2010 is an all new and more powerful WCF-SAP adaptor. When my colleagues tested out the new adaptor they discovered that the format of the data extracted from SAP was not identical to the old adaptor. This is not a big deal if the structure of the messages from SAP is simple. In this case we were receiving the delivery and invoice iDocs. Both these structures are complex especially the delivery document. Over the past few years I have tweaked the delivery mapping to remove bugs from original mapping. The idea of redoing these maps did not appeal and due to the current work load was not even an option. I opted for a rather crude alternative of pulling in the iDoc in the new typed format and then adding a static map at the start of the orchestration to convert the data to the old schema.

Note WCF-SAP data formats (on the binding tab of the configuration dialog box is the ‘RecieiveIdocFormat’ field):

Typed: Returns a XML document with the hierarchy represented in XML and all fields being represented by XML tags.
RFC: Returns an XML document with the hierarchy represented in XML but the iDoc lines in flat file format.
String: This returns the iDoc in a format that is closest to the original flat file format but is still wrapped with some top level XML tags. The files also contained some strange characters at the end of each line.

I started with the invoice document and it was quite straight forward to add the mapping but this is where my problems started. The orchestrations for these documents are dynamic and so require the identity of the partner to be able to correctly configure the orchestration. The partner identity is in the EDI_DC40 segment of the iDoc. In the old project the RECPRN node of the segment was promoted. The code to set a variable to the partner ID was now failing. After lot of head scratching I discovered the problem was due to the addition of Namespaces to the fields in the EDI_DC40 segment. To overcome this I needed to use an xPath query with a Namespace Manager. This had to be done in custom code.

I now tried to repeat the process with the delivery document. Unfortunately when we tried to get sample typed data from SAP an exception was thrown.

The adapter "WCF-SAP" raised an error message. Details "Microsoft.ServiceModel.Channels.Common.XmlReaderGenerationException: The segment or group definition E2EDKA1001 was not found in the IDoc metadata. The UniqueId of the IDoc type is: IDOCTYP/3/DESADV01/ZASNEXT1/640. For Receive operations, the SAP adapter does not support unreleased segments.

Our guess is that when the WCF-SAP adaptor tries to down load the data it retrieves a data schema from SAP. For some reason the schema does not match the data. This may be due to the version of SAP we are running or due to a customization. Either way resolving this problem did not look easy.

When doing some research on this problem I found an article showing me how to get the data from SAP using the WCF-SAP adaptor without any XML tags.

http://blogs.msdn.com/b/adapters/archive/2007/10/05/receiving-idocs-getting-the-raw-idoc-data.aspx

Reproduction of Mustansir blog:

Since the WCF based SAP Adapter is ... well, WCF based, all data flowing in and out of the adapter is encapsulated within a SOAP message. Which means there are those pesky xml tags all over the place. If you want to receive an Idoc from SAP, you can receive it in "Typed" format (in which case each column in each segment of the idoc appears within its own xml tag), or you can receive it in "String" format (in which case there are just 2 xml tags at the top, the raw xml data in string/flat file format, and the 2 closing xml tags).

In "String" format, an incoming idoc (for ORDERS05, containing 5 data records) would look like:

<ReceiveIdoc xmlns='http://Microsoft.LobServices.Sap/2007/03/Idoc/'><idocData>EDI_DC40 8000000000001064985620 E2EDK01005 800000000000106498500000100000001 E2EDK14 8000000000001064985000002000000020111000 E2EDK14 8000000000001064985000003000000020081000 E2EDK14 80000000000010649850000040000000200710 E2EDK14 80000000000010649850000050000000200600</idocData></ReceiveIdoc>

(I have trimmed part of the control record so that it fits cleanly here on one line).

Now, you're only interested in the IDOC data, and don't care much for the XML tags. It isn't that difficult to write your own pipeline component, or even some logic in the orchestration to remove the tags, right? Well, you don't need to write any extra code at all - the WCF Adapter can help you here!

During the configuration of your one-way Receive Location using WCF-Custom, navigate to the Messages tab. Under the section "Inbound BizTalk Messge Body", select the "Path" radio button, and:

(a) Enter the body path expression as: /*[local-name()='ReceiveIdoc']/*[local-name()='idocData']

(b) Choose "String" for the Node Encoding.

What we've done is, used an XPATH to pull out the value of the "idocData" node from the XML. Your Receive Location will now emit text containing only the idoc data.

You can at this point, for example, put the Flat File Pipeline component to convert the flat text into a different xml format based on some other schema you already have, and receive your version of the xml formatted message in your orchestration.

This was potentially a much easier solution than adding the static maps to the orchestrations and overcame the issue with ‘Typed’ delivery documents. Not quite so fast…

Note: When I followed Mustansir’s blog the characters at the end of each line disappeared.

After configuring the adaptor and passing the iDoc data into the original flat file receive pipelines I was receiving exceptions.

There was a failure executing the receive pipeline: "PAPINETPipelines.DeliveryFlatFileReceive, CustomerIntegration2.PAPINET.Pipelines, Version=1.0.0.0, Culture=neutral, PublicKeyToken=4ca3635fbf092bbb" Source: "Pipeline " Receive Port: "recSAP_Delivery" URI: "D:\CustomerIntegration2\SAP\Delivery\*.xml" Reason: An error occurred when parsing the incoming document: "Unexpected data found while looking for:

'Z2EDPZ7'

The current definition being parsed is E2EDP07GRP. The stream offset where the error occured is 8859. The line number where the error occured is 23. The column where the error occured is 0.".

Although the new flat file looked the same as the old one there was a differences. In the original file all lines in the document were exactly 1064 character long. In the new file all lines were truncated to the last alphanumeric character.

The final piece of the puzzle was to add a custom pipeline component to pad all the lines to 1064 characters. This component was added to the decode node of the custom delivery and invoice flat file disassembler pipelines.

Execute method of the custom pipeline component:

public IBaseMessage Execute(IPipelineContext pc, IBaseMessage inmsg)

{

//Convert Stream to a string

Stream s = null;

IBaseMessagePart bodyPart = inmsg.BodyPart;

// NOTE inmsg.BodyPart.Data is implemented only as a setter in the http adapter API and a

//getter and setter for the file adapter. Use GetOriginalDataStream to get data instead.

if (bodyPart != null)

s = bodyPart.GetOriginalDataStream();

string newMsg = string.Empty;

string strLine;

try

{

StreamReader sr = new StreamReader(s);

strLine = sr.ReadLine();

while (strLine != null)

{

//Execute padding code

if (strLine != null)

strLine = strLine.PadRight(1064, ' ') + "\r\n";

newMsg += strLine;

strLine = sr.ReadLine();

}

sr.Close();

}

catch (IOException ex)

{

throw new Exception("Error occured trying to pad the message to 1064 charactors");

}

//Convert back to stream and set to Data property

inmsg.BodyPart.Data = new MemoryStream(Encoding.UTF8.GetBytes(newMsg)); ;

//reset the position of the stream to zero

inmsg.BodyPart.Data.Position = 0;

return inmsg;

}

BizTalk 2010, BAM, SharePoint and the Default Application Pool Flag

Geordie — Tue, 17 Aug 2010 20:06:27 GMT

I have recently started to look at EDI communications through BizTalk. Part of the EDI features in BizTalk is the use of BAM to track messaging. Amongst the applications we currently have built in BizTalk we do some SharePoint communication. This is also a possible component to our EDI solution. We also have plans to move to BizTalk 2010 in the near future, so today I started to build my new BizTalk Dev machine.

I got a bit of a surprise when 2 parts of the required set had me setting a flag “Enable 32-bit applications” to opposite values.

First configuration: I was working the setup document available from Microsoft ‘Installing BizTalk Server 2010 on Windows Server 2008 R2 and 2008.docx’.

On pg 13…
Enable Internet Information Services
Microsoft Internet Information Services (IIS) provides a Web application infrastructure for many BizTalk Server features. BizTalk Server requires IIS for the following features:
• HTTP adapter
• SOAP adapter
• Windows SharePoint Services adapter
• Secure Sockets Layer (SSL) encryption
• BAM Portal
To enable Internet Information Services 7.5
1. Click Start, point to Administrative Tools and then click Server Manager.
2. ….Step by step instruction here
Note
BAM Portal runs only runs on a 32-bit mode. If you are installing IIS on a 64-bit machine then you must ensure that ASP.NET 2.0 is enabled on 32-bit mode. To do this, follow these steps:
1. On the taskbar, click Start, point to Administrative Tools, and then click Internet Information Services (IIS) Manager.
2. In the Connections pane, expand the server name, and then click Application Pools.
3. In the Actions pane, click Set Application Pool Defaults...
4. On the Application Pool Defaults dialog box, in Enable 32-bit applications, select True.

I set the Enable 32-bit applications flag as specified and continued with the install.

Second configuration: As mentioned we use SharePoint and to use the SharePoint adaptor we need to install the SharePoint Foundation 2010 or WSS3(SP2) components. After installation and configuration the SharePoint Sites where inaccessible and returned a HTTP error 503 ‘Site unavailable’ message. The application pools for the SharePoint components were stopped each time I tried to access the SharePoint sites. After a bit of searching I found that to get the site to work I needed to set the ‘Enable 32-bit applications, to False’.

Resolution: I was able to resolve the issue by setting the default application Pool’s Enable 32-bit applications to true as described in the text. I then went to each SharePoint application pool and right clicked. In the context menu there is an ‘Advanced Settings…’ option. Selecting this allowed me to set the ‘Enable 32-bit Applications’ flag for that specific pool. I was then able to access the SharePoint site even though the Default Application Pool's flag was set to 'true'.

Hope this helps, Geordie...

Note: When I tried to configure the BizTalk server using the basic configuration tool it failed badly. Once I gave the BizTalk service account create permissions on the database I was able to automaticaly configure BizTalk up to the BAM components. after that I had to manually configure each piece, setting the 'default application pool' appropriatly for the componet I was configuring. Also when configuring the SharePoint adaptor you need to select the correct website (ie the 'SharePoint-80' site from the dropdown list, the default web site will not work).

Adding Custom Pipeline code to the Pipeline Component Toolbox

Geordie — Thu, 25 Mar 2010 18:55:58 GMT

Add ...

"C:\Program Files\Microsoft Visual Studio 8\SDK\v2.0\Bin\GacUtil.exe" /nologo /i "$(TargetPath)" /f
xcopy "$(TargetPath)" "C:\Program Files\Microsoft BizTalk Server 2006\Pipeline Components" /d /y

to the 'Post Build event command line' for the pipeline support component project in visual studio. This will automatically put the support dll in the GAC and BizTalk’s Pipeline component directory on building the solution/project.

Build the project.

Right Click on title bar and select Choose items. Click on the BizTalk Pipeline Components tab and select the new component from the list. If it is not available in the list Browse to the BizTalk pipeline component folder (C:\Program Files\Microsoft BizTalk Server 2006\Pipeline Components) and select the dll.

BizTalk DB Log File Explosion Due to Corrupt Index

Geordie — Wed, 17 Jun 2009 23:24:19 GMT

I have just encounted an unusual database problem. My dev BizTalk server suddenly ran out of space. Initially I thought it was the transaction logs. No luck there they were well within normal limits. I did check the size ot the
'Microsoft SQL Server' directory. It was over 8GB. The volume was traced to the LOG directory. SQL was pumping out 4 MB of error log every 10 seconds. After fighting with the server to free up some disk space, I checked out the error files. The parts appear to be:

  ***Stack Dump being sent to C:\Program Files\Microsoft SQL Server\MSSQL.1\MSSQL\LOG\SQLDump7243.txt
  * *******************************************************************************
  *
  * BEGIN STACK DUMP:
  *   06/17/09 15:52:15 spid 54
  *
  * CPerIndexMetaQS::ErrorAbort - Index corruption
  *
  * Input Buffer 510 bytes -
  *             EXECUTE msdb.dbo.sp_sqlagent_log_jobhistory @job_id = 0xBE810
  * 129F3D1574AA4EB72EE897A6CCC, @step_id = 0, @sql_message_id = 0, @sql_sev
  * erity = 0, @run_status = 1, @run_date = 20090617, @run_time = 155201, @r
  * un_duration = 3, @operator_id_emailed = 0, @operator_id_netsent = 0, @op
  * erator_id_paged = 0, @retries_attempted = 0, @session_id = 65, @message
  * = N'The job succeeded. The Job was invoked by Schedule 4 (Schedule). T
  * he last step to run was step 1 (Purge).'

And

  2009-06-17 15:52:14.55 spid56      Error: 8646, Severity: 21, State: 1.
  2009-06-17 15:52:14.55 spid56      Unable to find index entry in index ID 1, of table
  117575457, in database 'msdb'. The indicated index is corrupt or there is a
  problem with the current update plan. Run DBCC CHECKDB or DBCC CHECKTABLE. If
  the problem persists, contact product support.
  2009-06-17 15:52:15.24 spid54 Using 'dbghelp.dll' version '4.0.5'
  2009-06-17 15:52:15.24 spid54      **Dump thread - spid = 54, PSS = 0x0DDD32D0, EC = 0x0DDD32D8
  2009-06-17 15:52:15.24 spid54      ***Stack Dump being sent to C:\Program Files\Microsoft SQL
  Server\MSSQL.1\MSSQL\LOG\SQLDump7243.txt
  2009-06-17 15:52:15.24 spid54      * *******************************************************************************
  2009-06-17 15:52:15.24 spid54      *
  2009-06-17 15:52:15.24 spid54      * BEGIN STACK DUMP:
  2009-06-17 15:52:15.24 spid54      *   06/17/09 15:52:15 spid 54
  2009-06-17 15:52:15.24 spid54      * 2009-06-17 15:52:15.24 spid54      * CPerIndexMetaQS::ErrorAbort - Index corruption

Searching for 'CPerIndexMetaQS::ErrorAbort - Index corruption' online led me to a post that suggested running the following SQL script from SQL Server Manager on the local machine.

  DECLARE tempcursor
  CURSOR
  READ_ONLY
  FOR
        select spid From master..sysprocesses
        where dbid = (select database_id from sys.databases where name = 'msdb')

  DECLARE @name int
  OPEN tempcursor

  FETCH NEXT FROM tempcursor INTO @name
  WHILE (@@fetch_status <> -1)
  BEGIN
        exec('kill ' + @name)

        FETCH NEXT FROM tempcursor INTO @name
  END

  Use msdb
  Go
  sp_dboption 'msdb', 'single_user', 'true'
  GO
  DBCC CHECKDB('msdb', REPAIR_REBUILD)
  go
  sp_dboption 'msdb', 'single_user', 'false'

  CLOSE tempcursor
  DEALLOCATE tempcursor

The script found and repaired the corrupt index, as shown below.

  DBCC results for 'sysdownloadlist'.
  There are 0 rows in 0 pages for object "sysdownloadlist".
  DBCC results for 'sysjobhistory'.
  Repair: Successfully deleted row in index "dbo.sysjobhistory, nc1" in database "msdb".
  Msg 8952, Level 16, State 1, Line 1
  Table error: table 'sysjobhistory' (ID 117575457). Index row in index 'nc1' (ID 2) does not match any data row. Possible extra or invalid keys for:
          The error has been repaired.
  Msg 8956, Level 16, State 1, Line 1
  Index row (1:1141:114) with values (job_id = '89F8A96E-AF2B-4BE9-B5E4-FEF3B8C94EC5' and instance_id = 466459) pointing to the data row identified by (instance_id = 466459).
  There are 6640 rows in 459 pages for object "sysjobhistory".
  CHECKDB found 0 allocation errors and 1 consistency errors in table 'sysjobhistory' (object ID 117575457).
  CHECKDB fixed 0 allocation errors and 1 consistency errors in table 'sysjobhistory' (object ID 117575457).

The log file generation has stopped and the server appears to be back to normal.

How to Map the Selection of the First Item in a Collection to the Final Document

Geordie — Wed, 13 May 2009 16:04:57 GMT

I recently came across a new mapping problem. I needed to make sure that the final document only had one address for a paper mill to satisfy the requirement of a standard documentation format (PAPINet). Usually the product is all made to order but every so often paper may be available from stock that was made in a different mill. The SAP system providing the information contains information on both mills. The challenge here was to remove the additional mills and only include the mill data for the first mill. In this case the mill information is not important as the ID for the paper (TAPI) includes a character that identifies the producing mill.

My first attempt was to limit the number of occurrences of the Mill element. BizTalk ignored the limit.

I then tried to use the 'Data Flattening' functoids with an 'Iteration' functoid. I set the 'Equal' functoid to receive the value from the 'Iteration' functiod and added a constant value of 1 (BizTalk must use 1 base in it's iterations). When the 'Iteration' functoid equals 1 it will resolve to true allowing the 'Data Flattening' functoids to only process the first occurrence of the Mill attribute.

The overall solution is a complex mapping from SAP to PAPINet. It uses a canonical structure to assist in the transformation. This problem was greatly simplified by using the canonical to canonical mapping (added to allow some complex calculations to be preformed).

How to block the creation of parent nodes when using the data flattening functoid

Geordie — Thu, 12 Mar 2009 20:17:27 GMT

I've had a few BizTalk maps that have caused me to stop and think as to how to get over the problem of suppressing nodes when the child node has no data. The issue arises if you use Data flattening shape to block the creation of an element. The child node is suppressed but the parent node is still created. I just found out how to block the parent node from being created so I’m posting it here so I can find it again the next time I need it.

By connecting the parent nodes to the functoid that provides the condition to the data flattening functoids the parent nodes will be suppressed.

Thanks to Saravana Kumar for posting the answer on his blog.

Geordie

Installing custom functoids

Geordie — Wed, 21 Jan 2009 22:46:51 GMT

I just got bit by this one a second time. So it is time to store the information here to avoid a third occurrence.

If you get an exception that the functoid was not found.

For example:- 'Exception Caught: Functoid not found:
guid({5DE500CC-45BC-454b-A23D-24449899042C})  funcid(6123)'

Check that the dll is installed in the mapper extension folder 'C:\Program Files\Microsoft BizTalk Server 2006\Developer Tools\Mapper Extensions' as well as the GAC.

Geordie

Problems receiving a Multi Part Message via a HTTP vs File Adaptor

Geordie — Tue, 13 Jan 2009 19:06:01 GMT

I recently built a recieve location that used used a File adaptor to test the recieve and Orchestration for a new multipart message. It all went well until I converted the recieve port to a HTTP adaptor. At that point my custom component in the recieve pipeline raised a 'method not implemented' error.

After several tests I was sure that the error was specific to HTTP adaptor and was not a problem with the installation of the custom component. One theory that I had was that an aditional method(s) may be required when implementing a custom components in a HTTP recieve pipeline hence the error. This was proven to be wrong.

In fact the issue was due to less methods being implemented in the HTTP adapter API. In this case I was calling a property setter that was implemented in the File adaptor API but not in the HTTP adaptor API. Specifically I was calling msg.BodyPart.Data getter.

To overcome this problem we used the get OriginalDataStream method to retieve the message stream as shown in the code below.

        public IBaseMessage Execute(IPipelineContext pc, IBaseMessage inmsg)
        {
            //Convert Stream to a string
            Stream s = null;
            IBaseMessagePart bodyPart = inmsg.BodyPart;

            // NOTE inmsg.BodyPart.Data is implemented only as a setter in the http adapter API and a
            //getter and setter for the file adapter. Use GetOriginalDataStream to get data instead.
            if (bodyPart != null)
                s = bodyPart.GetOriginalDataStream();

            byte[] bytes = new byte[s.Length];
            int n = s.Read(bytes, 0, (int)s.Length);
            string msg = new ASCIIEncoding().GetString(bytes).TrimEnd(null);
            //Get boundry value from first line of code
            string boundry = msg.Substring(2, msg.IndexOf("\r\n")-2);
            //Create new start to message with MIME requirements.
            msg = "MIME-Version: 1.0\r\nContent-Type: multipart/related; boundary=\"" + boundry + "\"\r\n" + msg;
            //Convert back to stram and set to Data property
            inmsg.BodyPart.Data = new MemoryStream(Encoding.UTF8.GetBytes(msg)); ;

            //reset the position of the stream to zero
            inmsg.BodyPart.Data.Position = 0;
            return inmsg;
        }

(Note: to debug the custom code in the HTTP recieve pipeline attach to the w3wp.exe process)