BizTalk

BizTalk FTP send adapter and Unix

mattjgilbert — Wed, 03 Nov 2010 10:50:33 GMT

This may well be already known but it took us a little while to figure out.

When using the BizTalk FTP adapter to write files to a Unix FTP server you may have to ensure the recipient of those files has the right to read/delete them.

When you ftp files, the master ftp configuration file determines the permissions that those files will have. You can set the permissions yourself by using a version of the umask command in the Before Put property of your FTP send port.

e.g. site umask 000

This will give everyone permission to do anything with the files. Other numerical values will have different effects (see here: http://en.wikipedia.org/wiki/Umask).

Equally of course, anyone writing files for you to pick up will likely have to do the same.

High Availability

mattjgilbert — Thu, 27 May 2010 10:58:56 GMT

Udi Dahan presented at the UK Connected Systems User Group last night. He discussed High Availability and pointed out that people often think this is purely an infrastructure challenge. However, the implications of system crashes, errors and resulting data loss need to be considered and managed by software developers. In addition a system should remain both highly reliable (backwardly compatible) and available during deployments and upgrades. The argument is that you cannot be considered highly available if your system is always down every time you upgrade.

For our recent BizTalk 2009 upgrade we made use of our Business Continuity servers (note the name, rather than calling them Disaster Recovery servers  ) to ensure our clients could continue to operate while we upgraded the Production BizTalk servers. Then we failed back to the newly built 2009 environment and rebuilt the BC servers. Of course, in the event of an actual disaster there was a window where either one or the other set were not available to take over – however, our Staging machines were already primed to switch to production settings, having been used for testing the upgrade in the first place.

While not perfect (the failover between environments was not automatic and without some minimal outage) planning the upgrade in this way meant BizTalk was online during the rebuild and upgrade project, we didn’t have to rush things to get back on-line and planning meant we were ready to be as available as we could be in the event of an actual disaster.

VSS 2005 error with BizTalk projects in VS 2008

mattjgilbert — Mon, 25 Jan 2010 16:25:42 GMT

We’ve been upgrading our BTS 2006 R2 projects to Visual Studio 2008 while using Visual SourceSafe 2005. We all started to get this annoying error pop-up when checking files in and out.

Unexpected error encountered. It is recommended that you restart the application as soon as possible.
Error: Not implemented
File: vsee\pkgs\vssprovider\csolutionnodebase.cpp
Line number: 2111

There is a workaround available thanks to Stephen Burdeau

In the VSS "Check Out for Edit" or "Check In" dialog, change from the default "Tree View" to "Flat View". To do this, click the corresponding toolbar button on the dialog.

Fix for POP3 adapter

mattjgilbert — Thu, 15 Oct 2009 09:55:20 GMT

A couple of posts back, I described an issue we had with emails not being downloaded by the BizTalk POP3 adapter. I'm happy to say that after working with Microsoft, a hotfix has been created for this which I tested yesterday. It'll take a few weeks for the KB article to be written up but I'll post details as soon as I have them. This fixes the "The POP3 adapter received a response line from a server that contains more than 512 characters" issue.

EDIT... some good news. It looks like the fix will find its way into BTS 2006 R2 SP1. The KB number will be 975826

EDIT 2: Here's the kb link: http://support.microsoft.com/kb/975826/en-us

Moving where rules execute

mattjgilbert — Wed, 20 May 2009 15:11:16 GMT

We have a ticketing application deployed to BizTalk which helps automate the workflow around customer updates and the sending of emails. One of the functions the application performs is to handle returned emails (out of office and non-delivery) and update the ticket with that information. However, the implementation has caused us some issues.

Right now, emails sent from the application are sent from the support mail account. In order to get any returned emails, all emails arriving in the support mailbox are copied to a second mailbox which BizTalk polls. Rules called from within an orchestration built to process these returned mails determine if an email is an Out of Office message or a NDR and if so, updates the originating ticket. If the email is neither of these, the message is discarded.

This has historically caused many issues. All spam and non-ticket related email hitting the support mailbox is copied and processed by BizTalk. On top of that, many messages cause decoding errors (S/MIME or MIME type problems for example) and the POP adapter itself can even bomb due to an error receiving badly formatted messages (causing a bottleneck until the offending email is cleared out the mailbox). Others have had this error too ("The POP3 adapter received a response line from a server that contains more than 512 characters" http://www.biztalkgurus.com/forums/t/5992.aspx). Running some stats on the emails BizTalk was having to process, we figured that less than 1% of the messages we were getting were the OOO or NDR ones we were interested in.

I have proposed we move where the rules are executed and put them into the support mailbox itself. Instead of having a blanket copy-all rule to the secondary mailbox, we should only copy over OOO and NDR messages. The rules will execute in an environment designed and optimized for handling and routing emails (Exchange) and processing and errors will be dramatically reduced on the BizTalk server. Being able to change rules without recompiling code is great but being able to move where they are executed is nice too.

Thanks

mattjgilbert — Tue, 05 May 2009 16:55:49 GMT

Thanks to Solidsoft for running a great day at Microsoft last week. Topics covered were BizTalk, SharePoint and using them to build successful SOA and BPM solutions.

Being responsible for BizTalk and also having had SharePoint land on my plate recently, the day gave me some good ideas about where to take these platforms.

Consolidated blog resource

mattjgilbert — Wed, 08 Apr 2009 15:30:35 GMT

Here is a really useful consolidation of blog posts: http://blogdoc.biztalk247.com/

It might save searching around for some answers :)

A brief note on Canonical Schemas

mattjgilbert — Tue, 07 Apr 2009 20:48:21 GMT

We used the model of canonical schemas as lot at my previous company. In fact we used it pretty much everywhere and retro-fitted older Integration solutions to use canonical schemas wherever and whenever we could. They buy you a lot and it’s a model I’m going to try and introduce to my current workplace.

If you are really lucky, your company might already have a notion of common business entities and a “data policing” department which controls what they look like. If so you already know what your canonical schemas are going to look like. If not, no matter; the key thing is that having an “Internal” schema inside of BizTalk provides a level of indirection or decoupling from the representations of business entities in the outside world. In an ideal world, all messages passing between systems within an Enterprise use the same schema and transformation is minimal or non- existent. B2B communication using industry defined schemas also minimises this. However, we very seldom find our selves in an ideal world and transformation at the middleware boundary is a fact of life for us.

Creating your “canonical” view of what an entity looks like and implementing it within your middleware can help insulate you from changes in the future. If you build an orchestration which directly uses an external schema, you are more susceptible to changes to that schema. If your orchestration handles a canonical message however, you may only have to update the transformation and continue to publish the canonical message to the MessageBox without further changes to any subscribing orchestrations. This really shows its worth if you have multiple subscribers to a single published message as you may only have a single change to make.

Canonical schemas also help developers understand business entities and provide consistency across solutions and integrations as they offer the opportunity for reuse. If you’ve defined what your company considers to be an “Invoice”, the same schema applies to all integration solutions. They can use meaningful names for elements and attributes rather than system or domain specific ones which may be alien to your organisation or someone not used to the sending platform (SAP schemas are a great example!). Referencing plays an important part also, and canonical schemas can reference other ones (a Canonical Invoice schema might reference a Canonical Address schema for example).

I prefer to keep all Canonical schemas together in their own containing application within BizTalk (<companyname>.Canonical.Schemas). I also like having an “Enterprise Header” applied to each schema which promotes content for routing and tracking. I’ll discuss that in another post.

Pipeline handling of SOAP faults

mattjgilbert — Fri, 13 Mar 2009 19:24:11 GMT

I’ve recently been working a fair bit with WCF and web services in and out of BizTalk. One of the things this involves is hooking up some orchestration ports using the generated port types when the target service is consumed. Once the consuming wizard is done, the generated port type definitions can then be edited to include a fault message. I chose to assign the out-the-box BTS.soap_envelope_1__1+Fault message.

Now, the physical 2-way send port the orchestration wires up to has a custom receive pipeline which includes an XML disassembler and a collection of explicitly named document schemas the pipeline can disassemble. I made sure that BTS.soap_envelope_1__1+Fault was one of those schemas that should be recognised. When a good response message was returned, all was well but when the target service returned a fault, the pipeline spat it out with the classic “No Disassemble stage components can recognize the data”.

Other pipelines in other projects were working in exactly the same way and were receiving and dealing with their fault messages perfectly fine. So what was wrong? After much solution cleaning/ rebuilding (hoping it was just an assembly caching issue) and getting nowhere it took a step back and looked again at a working pipeline. The only difference was that in the list of document schemas set on the XML disassembler, the fault message was the first in the list on the working pipeline and the last in the list on my one. Sure enough, when I reordered the list to put the fault message at the top and redeployed it, fault messages were then disassembled correctly.

This rang some vague bells in my head at the time to do with first matching in the list but I can’t see why a first match would fail if the only match happened to be at the bottom of the list – it’s still there and still the first match.

String splitting

mattjgilbert — Tue, 24 Feb 2009 15:37:37 GMT

I was asked the other day about whether it was possible to split a string value into constituent parts based on a delimiter. The requirement was to split out the original parts of a code into separate line items in a message.

e.g. <original_code>xxx/yyy/123</original_code>

becomes

<line_item>xxx</line_item>
<line_item>yyy</line_item>
<line_item>123</line_item>

Richard Seroter blogged about this a while back. You can find his solution here.

Also, if you need to do the reverse (take multiple values in repeating records and make them a single delimited value), Stephen Kaufman blogged about this recently. You can find that here.