Michael Stephenson

Twitter API 401 Unauthorized

Michael Stephenson — Sat, 17 Dec 2011 22:24:11 GMT

Ive been doing a little stuff with the Twitterize library and the twitter API.

I started unexpectedly getting a 401 unauthorized error when things had been working fine previously.

Eventually I tracked this down to be an issue with clock sync. I am using VM Ware fusion on Mac and when if suspended then resumed my VM the clock wasnt always in sync. When this was fixed it all works fine.

Note to self the way to change the VM is

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1021718

BizTalk Solution Analyser

Michael Stephenson — Sat, 10 Dec 2011 17:18:21 GMT

For a while I’ve been working on an idea and whitepaper around analyzing source code to measure the size and complexity. There are some tools around which can do this but my differentiator is that I was interested in BizTalk projects and wanted to look at how these could be measured. Note however this can be easily used for non-BizTalk projects too.

The aim is to be able to understand the size and complexity of a solution and to allow some real metrics to contribute to your understanding of the Total Cost of Ownership of your solution. Using the tools I have developed alongside this I have been working with some customers to integrate the analysis into the build and continuous integration processes so that we can produce a report of the code and also track changes over time to monitor how the code based vary in terms of size and complexity.

The white paper goes into the detail of the theory about how we measure code and is available on the following link:

https://skydrive.live.com/redir.aspx?cid=983a58358c675769&resid=983A58358C675769!1820&parid=983A58358C675769!1819

The source code and application/tools which can do this for you are available on the following codeplex project site

http://biztalksolutionanaly.codeplex.com/

At this stage the project is being used to monitor approximately 20 code bases at two companies who have significant investments in BizTalk and I’m also looking for feedback from the community over other things we should measure.

It would be great to hear how you get on with this or if you have any views on it.

Labelling a Build in TFS from Cruise Control

Michael Stephenson — Fri, 04 Nov 2011 22:11:35 GMT

This is just a reminder for my self of how and why we do this.

We have 2 projects within a TFS project collection for our integration component developments. We have:

1. A project for .net based integration projects

2. A project for BizTalk based integration projects

The main reason we do this is so we dont have loads of TFS projects as we have a significant number of components but also we want some different rules around check in and source control locks etc etc.....

At this stage our build servers are still running cruise control rather than Team Build so one of the problems i was finding was when cruise control labelled a build in TFS i couldnt tell easily in the label search which component or folder the build label related to. We modified the build scripts to do a custom label with TF.exe so that we could put a specific label from each build. We used the following command:

Using this command means that the build label is applied at the right place for the workspace but also the label includes the cruise control project name making it easy to workout what the label was actually for rather than just being a bunch of version numbers.

There is probably a better way to do this but its working fine for us.

BizTalk 360 Success Story

Michael Stephenson — Wed, 19 Oct 2011 21:51:29 GMT

We are currently evaluating BizTalk 360 at one of my clients at the moment and I plan to write a more detailed post in the future about our experiences but in the meantime I wanted to make a comment about something this week which was particularly useful.

We have a very large project with many teams and vendors and in our first integration test environment all of the delivery teams do daily deployments to this test environment. From here a successfully tested release could be promoted to other environments as required. Its quite painful at times due to the pace of change in this first test environment and im sure most readers would be familiar with the typical challenges around integration and biztalk being the bit that identifies the problems and often not be the cause of the problem. Yes you know what I am talking about.

A significant amount of my teams time is taken up with troubleshooting issues and working out retrospectively where a problem came from. In our evaluations of BizTalk 360 one of the things I have done is to setup the BizTalk 360 health check emails to be published 4 times per day.

This week we had a situation where the test teams had signed off a release but had not realised for some reason there was an issue where some messages were unable to be loaded into an application. This whole release had been signed off to go into the UAT environment and then an hour later the BizTalk 360 email came out and we could see these suspended messages. We were able to identify and rectify this issue in the application which was not accepting messages and plug the gap in the test scripts before the release went to UAT.

Finding this issue in the system test environment and resolving it would have saved a significant impact with the problems this issue could have caused in the UAT environment in terms of lost test time etc etc. In my opinion this one situation probably paid for the cost of the BizTalk 360 license on its own for our large scale project so hopefully our evaluation will be finished soon and we will be able to roll this out across other environments.

Thanks Saravana keep up the good work

BDD BizTalk videos on CloudCast

Michael Stephenson — Fri, 05 Aug 2011 14:41:31 GMT

The biztalk videos for BDD and acceptance testing are now on cloudcast

http://www.cloudcasts.net/Default.aspx?category=BizTalk

Reminder to self - Terminator

Michael Stephenson — Tue, 19 Jul 2011 21:22:24 GMT

I always seem to spend ages looking for this when i need it so this is the link to terminator

http://www.microsoft.com/download/en/details.aspx?id=2846

Behaviour Driven BizTalk Development

Michael Stephenson — Mon, 18 Jul 2011 21:25:55 GMT

Ive recently done some user group sessions around BizTalk and Behaviour Driven Development and Acceptance Testing.

Ive uploaded the videos and samples for these sessions to the following codeplex site.

http://biztalkbddsample.codeplex.com/

They will also be on CloudCasts soon

Enjoy

Testing Inside BizTalk update

Michael Stephenson — Sun, 12 Jun 2011 00:14:27 GMT

Ive just released an update to the codeplex project so that there are test steps which are also compatible with BizUnit v4 which has recently been released http://btsloggingeventsinbi.codeplex.com/

Test whats happening inside BizTalk

Michael Stephenson — Wed, 18 May 2011 06:16:51 GMT

A few weeks ago I did the video about how we were testing what happens inside BizTalk by using the CAT Team Logging Framework and the ETW trace events and then testing the information coming out from our tracing to prove what was happening inside BizTalk.

Ive updated the codeplex project so that the code is a little easier for people to use and is packaged better. Also the source code is now available in codeplex and there is some additional documentation.

Note that it now works slightly differently to in the video but performs much better. The original video about the idea is on the following link

http://www.cloudcasts.net/ViewWebcast.aspx?webcastid=2521028145037920619

The codeplex project is on the following link:

http://btsloggingeventsinbi.codeplex.com/

My First Experience with BizTalk 360

Michael Stephenson — Sat, 14 May 2011 04:19:14 GMT

Ive recently just setup BizTalk 360 (http://www.biztalk360.com/) to try out the monitoring capabilities. I also thought it would be interesting to see how things go since this given environment was a BizTalk 2006R2 environment and fairly old stuff now.

I had a couple of issues during the install/setup but i was able to solve all of these using the troubleshooting page on the BizTalk 360 website and a little common sense. To be honest a couple of them were me being a bit lazy and not reading the install notes properly.

The changes I had to make were as follows:

1. Changed connection string to the biztalk 360 database as described on the troubleshooting website to integrated security

2. Change the IIS6 virtual directory to .net 4 rather than .net 2

3. Add the full trust element to the web.config file for BizTalk 360

4. Add the IIS NT Authentication providers as described on the troubleshooting page

Bearing in mind this was a CTP release I was very pleased with the support on the website which helped me to get this setup and even with these problems it still took less than 20 minutes. I would expect BizTalk 2006 to be a slightly less target for the BizTalk 360 team than BizTalk 2010 but it goes to prove it can be installed pretty easily and im not enjoying the management features which I think are a big opportunity for lots of organisations to really get their production operations of biztalk working in secure and effective way

Good job guys

BizTalk Build Generator & BizTalk 2010

Michael Stephenson — Sun, 24 Apr 2011 06:24:53 GMT

Ive eventually had time to migrate the build generator to work with BizTalk 2010.

Ive released this as a beta for now and would love to hear any feedback from others who might be using it while i work on getting some more testing with other migrated projects.

If your interested in taking a look please refer to:

http://biztalkmsbuildtool.codeplex.com/releases/view/65036

UK Connected Systems User Group - May 2011

Michael Stephenson — Fri, 15 Apr 2011 18:09:28 GMT

The next uk connected systems user group event is open for registration: 11th May 2011 @ EMC http://ukcsbugmay2011.eventbrite.com/

BizTalk RFID & NServiceBus

Michael Stephenson — Sat, 26 Mar 2011 07:18:25 GMT

Ive just posted a paper on the technet wiki about BizTalk RFID and NServiceBus used as a publishing component

http://social.technet.microsoft.com/wiki/contents/articles/biztalk-2010-amp-nservicebus.aspx

BizTalk Light & Easy Update

Michael Stephenson — Tue, 08 Mar 2011 14:35:59 GMT

Just to let everyone know the BizTalk Light and Easy Webcast Series has been updated with some new BizTalk 2010 videos

http://www.cloudcasts.net/Default.aspx?category=BizTalk+Light+and+Easy

Testing what is happening inside your BizTalk solution

Michael Stephenson — Tue, 01 Mar 2011 14:06:40 GMT

As BizTalk developers we all know that one of the common challenges is how to test your BizTalk solution once it is deployed to BizTalk. Hopefully most of us are using the BizUnit framework for testing, but we still have the limitation that it's a very Black Box test.

I have put together a sample and video to show a technique where I'm using the Logging Framework from the BizTalk CAT Team at Microsoft and where by BizUnit test is able to make assertions against the instrumentation going through the framework. This means that I can test for things happening such as the fact a component was executed or which branch of an orchestration was executed by simply using my normal instrumented code.

I've put the sample and video for this on the following codeplex site:

http://btsloggingeventsinbi.codeplex.com/

The video should also be on cloud casts fairly soon too.

CCNet TFS Migration - Dealing with left over folders

Michael Stephenson — Sat, 29 Jan 2011 05:30:32 GMT

Im currently in the process of migrating our many BizTalk projects from MKS source control to TFS. While we will be using TFS for work item tracking and source control etc we will be continuing to use Cruise Control for continuous integration although im updating this to CCNet 1.5 at the same time.

Ill post a few things as much as a reminder to myself about some of the problems we come across.

Problem

After the first build of our code the next time a build is triggered an error is encountered by the TFS source control block refreshing the source code.

System.IO.IOException: The directory is not empty.

   at System.IO.Directory.DeleteHelper(String fullPath, String userPath, Boolean recursive)
   at System.IO.Directory.Delete(String fullPath, String userPath, Boolean recursive)
   at ThoughtWorks.CruiseControl.Core.Sourcecontrol.Vsts.deleteDirectory(String path)
   at ThoughtWorks.CruiseControl.Core.Sourcecontrol.Vsts.GetSource(IIntegrationResult result)
   at ThoughtWorks.CruiseControl.Core.IntegrationRunner.Build(IIntegrationResult result)
   at ThoughtWorks.CruiseControl.Core.IntegrationRunner.Integrate(IntegrationRequest request)

System.IO.IOException: The directory is not empty. at System.IO.Directory.DeleteHelper(String fullPath, String userPath, Boolean recursive) at System.IO.Directory.Delete(String fullPath, String userPath, Boolean recursive) at ThoughtWorks.CruiseControl.Core.Sourcecontrol.Vsts.deleteDirectory(String path) at ThoughtWorks.CruiseControl.Core.Sourcecontrol.Vsts.GetSource(IIntegrationResult result) at ThoughtWorks.CruiseControl.Core.IntegrationRunner.Build(IIntegrationResult result) at ThoughtWorks.CruiseControl.Core.IntegrationRunner.Integrate(IntegrationRequest request)

Solution

The problem seems to be with a folder called TestLocations which is created by the build process and used along with the file adapter as a way to get messages into BizTalk. For some reason the source control block when it does a full refresh of the code does not get rid of this folder and then complains thats a problem and fails the build.

Interestingly there are other folders created by the build which are deleted fine. My assumption is that this if something to do with the file adapter polling the directory. However note that we have not had this problem with other source control blocks in the past.

To workaround this I have added a prebuild task to the ccnet.config file to delete this folder before the source control block is executed. See below for example

prebuild> exec>executable>cmd.exe</executable>buildArgs>/c "if exist "C:\<MyCode>\TestLocations" rd /s /q "C:\<MyCode>\TestLocations""</buildArgs>exec> prebuild>

BizTalk & Visual Studio Layer Diagrams

Michael Stephenson — Sun, 16 Jan 2011 05:38:50 GMT

Ive just recently done a video about BizTalk and Visual Studio 2010 Layer Diagrams which Alan Smith has kindly published on cloudcasts

http://www.cloudcasts.net/ViewWebcast.aspx?webcastid=2521072060555401694

UK Connected Systems User Group - Update and next meeting

Michael Stephenson — Thu, 23 Dec 2010 05:58:00 GMT

For those in the UK Connected Systems User Group, the content from our last meeting is in the below linked skydrive folder:
http://cid-40015ea59a1307c8.office.live.com/browse.aspx/.Public/SBUG/SBUG%20Meetings/2010%20December

We are currently preparing the next event for Tuesday 15th Febuary to again be at EMC in London Bridge. We are still formalising the details of the event but it is now open for registration.
http://ukcsugfeb2011.eventbrite.com/

Data Driven Unit Test - Excel DSN Error

Michael Stephenson — Sat, 13 Nov 2010 02:50:29 GMT

Just a little quick one for myself....

When running some data driven unit tests we were getting the below error.

Unit Test Adapter threw exception: System.Data.Odbc.OdbcException: ERROR [IM002] [Microsoft][ODBC Driver Manager] Data source name not found and no default driver specified.

The cause of this was that we had rebuild the machines and somehow when installing office the DSN for Excel hadnt been created. On all of the other machines it had been setup fine but for some reason on my machine it hadnt been setup.

The fix was to create a DSN called Excel Files pointing to excel drivers and then just rerun the unit tests.

As a reminder the test attributes for this were below:

[TestMethod][DeploymentItem(@"..\..\..\mappings.xlsx")]

[DataSource("System.Data.Odbc", "Dsn=Excel Files;dbq=|DataDirectory|\\spreadsheet.xlsx", "worksheet1$", DataAccessMethod.Sequential)]

Hosted TFS

Michael Stephenson — Sat, 06 Nov 2010 07:18:59 GMT

Just wanted to put forward a recommendation to anyone who may be interested in a hosted TFS solution.

As a regular traveller its quite tough to keep on top of some of my blog and other projects so for a while ive been thinking about TFS but to be honest I cant be bothered to setup and manage my own instance with all that it involved.

I came across the SaasMadeEasy offering for hosted TFS and have setup a basic subscription which is going to cost me $15 per month (as a single user).

Ive had a few conversations on email with Paul Hacker who is one of the main people behind this and they are really helpful and respond to any questions very quickly.

Im sure theres a few other people in a similar situation so if your thinking TFS definately check out these guys

http://www.saasmadeeasy.com/

Do people do BizTalk Versioning?

Michael Stephenson — Sun, 15 Aug 2010 09:08:55 GMT

Just wondering how many people actually do dynamic versioning of their BizTalk projects with Cruise Control or Team Build.

The reason I ask is you dont see that many things on the forums or blogs about it. I find that it can be quite challenging at times so Im not sure if we are just doing it the hard way or its something most people dont do?

Another little versioning Gotcha

Michael Stephenson — Sun, 15 Aug 2010 09:06:07 GMT

Ive been having some problems with versioning for BizTalk recently and in particular it was centred around one map.

Initially the problems were related to the configuration manager but then when this was resolved there was another little gotcha which is probably worth bearing in mind.

So on the development machine everything versions as 999.999.999.999 and this is usually pretty pain free. On the build server though a version number is applied from cruise control and replaces anything with 999.999.999.999 with the CCNet version number. Any other version number is not touched incase we are using multiple versions of one of our assemblies.

In the map I was having problems with it always seemed to pick up either an old version or version 0.0.0.0 which was not something that should have existed.

I eventually tracked the problem down the the xml under the map where you find the following snippet referencing your assembly.

What is happening is that because its a project reference between the map project and the C# project its popping in the path to the utilities assembly, however when I first created the map I must have been in Release configuration where as for the other maps I must have been in Debug configuration. In the development machine because its always the same version number this problem never shows up, but on the build server the version number changes with each build which makes this visible.

In the server build we do a Debug build and then a Release build. In the above case when we do the Debug build there is nothing in the the release folder so it was then using the 0.0.0.0 version number.

Unfortunately this issue doesnt display itself as a compilation error which is a big pain because you get to the BizUnit tests and find that you get strange failures which are a real problem to troubleshoot and its difficult to track this back to the source of the problem.

Anyway another one which might save someone a few hours

Unable to create the transform when your doing versioning on your build server

Michael Stephenson — Thu, 05 Aug 2010 00:13:43 GMT

I had a very painful experience this week on a project. We were implementing some new features in an existing BizTalk application and the first builds on the build server were good then suddenly a number of the tests started failing with the below error in the event log.

Event Type: Error
Event Source: XLANG/s
Event Category: None
Event ID: 10030
Date:  8/4/2010
Time:  1:36:21 PM
User:  N/A
Computer: ------------------
Description:
Uncaught exception (see the 'inner exception' below) has suspended an instance of service ------------------ImportMemberChange(88c96b70-f723-78d4-e8e1-d8464f189ee9)'.
The service instance will remain suspended until administratively resumed or terminated.
If resumed the instance will continue from its last persisted state and may re-throw the same unexpected exception.
InstanceId: 4bb8171d-ca71-4b9b-b8b8-5f0cf5fa6abd
Shape name: Ctor
ShapeId: ec543efe-e955-4860-a1b3-7b97dc4f3038
Exception thrown from: segment 1, progress 46
Inner exception: Error encountered while executing the transform ------------------.MemberImportMap. Error:Unable to create the transform..

Exception type: XTransformationFailureException
Source: Microsoft.XLANGs.Engine
Target Site: Void ApplyTransform(System.Type, System.Object[], System.Object[])
The following is a stack trace that identifies the location where the exception occured

   at Microsoft.XLANGs.Core.Service.ApplyTransform(Type mapRef, Object[] outParams, Object[] inParams)
   at --------------------------------.ImportMemberChange.segment1(StopConditions stopOn)
   at Microsoft.XLANGs.Core.SegmentScheduler.RunASegment(Segment s, StopConditions stopCond, Exception& exp)
Additional error information:

Could not load file or assembly ------------------.Utilities, Version=999.999.999.999, Culture=neutral, PublicKeyToken=------------------or one of its dependencies. The system cannot find the file specified.

Exception type: FileNotFoundException
Source: mscorlib
Target Site: System.Reflection.Assembly _nLoad(System.Reflection.AssemblyName, System.String, System.Security.Policy.Evidence, System.Reflection.Assembly, System.Threading.StackCrawlMark ByRef, Boolean, Boolean)
The following is a stack trace that identifies the location where the exception occured

   at System.Reflection.Assembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, Assembly locationHint, StackCrawlMark& stackMark, Boolean throwOnFileNotFound, Boolean forIntrospection)
   at System.Reflection.Assembly.nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, Assembly locationHint, StackCrawlMark& stackMark, Boolean throwOnFileNotFound, Boolean forIntrospection)
   at System.Reflection.Assembly.InternalLoad(AssemblyName assemblyRef, Evidence assemblySecurity, StackCrawlMark& stackMark, Boolean forIntrospection)
   at System.Reflection.Assembly.InternalLoad(String assemblyString, Evidence assemblySecurity, StackCrawlMark& stackMark, Boolean forIntrospection)
   at System.Reflection.Assembly.Load(String assemblyString)
   at Microsoft.XLANGs.BaseTypes.TransformBase.get_TransformArgs()
   at Microsoft.XLANGs.RuntimeTypes.TransformMetaData..ctor(Type transformBaseType)
   at Microsoft.XLANGs.RuntimeTypes.TransformMetaData._creator(Type t)
   at Microsoft.XLANGs.RuntimeTypes.MetadataCache._slowFor(Type t)
   at Microsoft.XLANGs.RuntimeTypes.MetadataCache.For(Type t)
   at Microsoft.XLANGs.RuntimeTypes.TransformMetaData.For(Type t)
   at Microsoft.XLANGs.Core.Service.ApplyTransform(Type mapRef, Object[] outParams, Object[] inParams)

Basically this error is saying that the assembly the map is using to reference some functions called by the scripting functoid can not be found.

There are many forum posts along the similar lines but our situation was a little different because the build would run fine when ran locally or on the build server outside of cruise control. It was only when cruise control runs the build and does some additional steps to apply versioning to the code that we saw this problem.

After lot of investigation I found the cause of the problem when I disassembled the map assembly using Reflector. With reflector you can see the XSLTAgruments property which contains a string which outlines the expansion objects which will be used by the transform when executing. This indicates the assemblies which will be called.

In this string I could see the wrong version number had been applied during the compile of the project it was just a question of working out why.

Again after a lot more digging I found that in our case it was my old friend the Visual Studio Configuration Manager which obviously decided that it hadnt been getting enough attention recently so decided to change the Debug/Release/DEployment/Development settings within the solution. Fortunately if you have BizTalk 2009 or are planning BizTalk 2010 you shouldnt get this problem but its a big pain for 2006R2. So what happened was that during the compilation the the BizTalk project is somehow picking up a reference to the default versioned utilities component rather than the version number supplied by cruise control.

I managed to fix this by going through all of the settings in the configuration manager and lining up the Debug --> Development and Release --> Deployment settings then everything just worked.

Im not sure how the settings get changed as it wasnt done intentionally by us and the solution file is source controlled but we had added some projects to the solution so it could have gotten in that way.

Anyway on the of chance it saves someone half a day figuring this out.

Connected Systems User Group Meeting tomorrow

Michael Stephenson — Wed, 26 May 2010 05:20:53 GMT

Just wanted to pass out a reminder about tomorrow nights user group meeting with Udi Dahan

Registration link:http://sbugmay2010.eventbrite.com/

User group link: http://ukconnectedsystemsusergroup.org

Multi-Part Map Troubleshooting

Michael Stephenson — Sat, 22 May 2010 15:53:10 GMT

Scenario

I came across a nice little one with multi-part maps the other day. I had an orchestration where I needed to combine 4 input messages into one output message like in the below table:

Input Messages

Output Messages

Company Details

Member Details

Event Message

Member Search

Member Import

I thought my orchestration was working fine but for some reason when I was trying to send my message it had no content under the root node like below

<ns0:ImportMemberChange xmlns:ns0="http://---------------/"></ns0:ImportMemberChange>

My map is displayed in the below picture.

I knew that the member search message may not have any elements under it but its root element would always exist. The rest of the messages were expected to be fully populated.

I tried a number of different things and testing my map outside of the orchestration it always worked fine.

The Eureka Moment

The eureka moment came when I was looking at the xslt produced by the map. Even though I'd tried swapping the order of the messages in the input of the map you can see in the below picture that the first part of the processing of the message (with the red circle around it) is doing a for-each over the GetCompanyDetailsResult element within the GetCompanyDetailsResponse message.

This is because the processing is driven by the output message format and the first element to output is the OrganisationID which comes from the GetCompanyDetailsResponse message.

At this point I could focus my attention on this message as the xslt shows that if this xpath statement doesn’t return the an element from the GetCompanyDetailsResponse message then the whole body of the output message will not be produced and the output from the map would look like the message I was getting.

<ns0:ImportMemberChange xmlns:ns0="http://---------------/"></ns0:ImportMemberChange>

I was quickly able to prove this in my map test which proved this was a likely candidate for the problem.

I revisited the orchestration focusing on the creation of the GetCompanyDetailsResponse message and there was actually a bug in the orchestration which resulted in the message being incorrectly created, once this was fixed everything worked as expected.

Conclusion

Originally I thought it was a problem with the map itself, and looking online there wasn’t really much in the way of content around troubleshooting for multi-part map problems so I thought I'd write this up.

I guess technically it isn't a multi-part map problem, but I spend a good couple of hours the other day thinking it was.