Charles Young

Friday, December 16, 2011 #

It's always exciting when a new application you've worked on goes live. The last couple of weeks have seen the 'soft' launch of a new service offered by the UK government called 'Tell Us Once' (TUO). You can probably guess from the name what the service does. Currently, the service allows UK citizens to inform the government (as opposed to Register Officers, who must still be notified separately) just once of two types of 'change-of-circumstance' event; namely births and deaths. You can go, say, to your local authority contact centre, where an officer will work through a set of screens with you, collecting the information you wish to provide. Then, once the Submit button is clicked, that's it! With your consent, the correct data sets are parcelled up and distributed to wherever they need to go - central and local government departments, public sector agencies such as the DVLA, Identity and Passport Service, etc. No need to write 50 letters!

With my colleagues at SolidSoft , I'm really proud to have been involved with the team that designed and developed this new service. For the past few years, we worked originally on the prototypes and pilots (there was more than one!). Over the last eighteen months or so, we have been engaged in building the national system, and development work in on-going. It's been a journey! The idea is very simple, but as you can imagine, the realisation of that idea is rather more complex. Look for future enhancements to today's service, with the ability to report events on-line from the comfort of your own home and the possible extension of the system to cover additional event types in future.

Interaction with government has just got a whole lot better for UK citizens, and we helped make that happen. It's a pity that I don't intend to have any more children (four is enough!), and I really hope I don't have to report a death in the near future, but if I do, I'll be beating a path to the door of my local council's contact centre in order to 'tell them once'.

See http://www.guardian.co.uk/government-computing-network/2011/dec/16/tell-us-once-matt-briggs?utm_source=twitterfeed&utm_medium=twitter

http://www.guardian.co.uk/public-leaders-network/2011/nov/10/tell-us-once-birth-death

Share This Post:

Posted On Friday, December 16, 2011 11:34 AM | Feedback (1)

Friday, December 09, 2011 #

BizTalk Server 2010 R2 Announced

Yesterday, Microsoft announced the forthcoming release of BizTalk Server 2010 R2 on the BizTalk Server blog site. This is advanced notice, given that this new version will ship six months after the release of Windows 8, expected in the second half of next year. On this basis, we can expect the new version of BizTalk Server to arrive in 2013. Given the BizTalk team’s previous record of name changes, I wonder if this will eventually be released as BizTalk Server 2013.

Microsoft has been refreshingly open in recent months about their future plans for BizTalk Server. This strategy has not been without its dangers with some commentators refusing to accept Microsoft’s statements at face value. However, yesterday’s announcement is entirely in line with everything Microsoft has been saying, both publically and privately, for some time now. Since the release of BizTalk Server 2004, Microsoft has made little change to the core technology with, of course, the exception of a much re-vamped application packaging approach in BizTalk Server 2006. Instead, Microsoft chose to put investment into a number of important ‘satellite’ technologies such as EDIFACT/X12/AS2 support, RFID Server, etc. Maintaining the stability of the core platform has allowed BizTalk Server to emerge as a mature and trusted workhorse in the enterprise integration space with widely available skills in the marketplace.

In terms of its major investments, Microsoft’s focus has long shifted to the cloud. Microsoft has candidly communicated that, given this focus, they have no current plans to add major new technologies to the BizTalk platform. In addition, they absolutely have no intention of re-engineering the core BizTalk platform. In my direct experience in recent months, this last point plays very well to prospective and existing enterprise customers. It takes us straight to the heart of what most organisations want from an integration server: a ‘known quantity’ with a good track record for dependability, scalability and stability and a significant pool of available technical resource.

The announcement of BizTalk Server 2010 R2 illustrates and illuminates Microsoft’s stated future strategy for the product. An important part of Microsoft’s platform for enterprise computing, it will continue to be enhanced and extended. It will match future developments in the Windows platform and new versions of Visual Studio. However, we should not expect to see any dramatic new developments in the world of BizTalk Server. Instead, the BizTalk platform will continue to steadily mature further as the world’s best-selling integration server.

One of the big messages of yesterday’s announcement is that BizTalk Server will increasingly support its emerging role in building hybrid solutions that encompass systems and services that reside both on-premises and in the cloud. At SolidSoft , we are increasingly focused on the design and implementation of cloud-based and hybrid integration solutions. Integration is challenging, and Azure is a young, fast evolving platform. Microsoft has discussed at length their vision of Azure within a wider ‘hybrid’ context. The availability of a tried and tested, mature, on-premises integration server is a vitally important enabler in building hybrid solutions. Better than that, the announcement makes it clear that, as well as new support for the Azure service bus, BizTalk Server 2010 R2 licensing will be revised to open up new opportunities for hosting the server in the cloud. This ties in with the push in Azure to embrace more fully the IaaS (infrastructure-as-a-service) model and, perhaps most importantly in the BizTalk space, to reduce or eliminate existing barriers between the on-premises and off-premises worlds. BizTalk Server and Azure belong together.

Share This Post:

Posted On Friday, December 09, 2011 8:37 AM | Feedback (0)

Sunday, September 25, 2011 #

Announcing BizTalk Server 2010 Unleashed

At last, I can announce that ‘BizTalk Server 2010 Unleashed’ has been published and is available through major booksellers in both printed and electronic form. The book is not a new edition of the old ‘BizTalk Server 2004 Unleashed’ book from several years ago, although Brian Loesgen, our fearless team leader, provided continuity with that title. Instead, this is entirely new content written by a team of six authors, including myself.

BizTalk Server is such a huge subject. It proved a challenge to decide on the content when we started our collaboration a couple of years back (yes, it really was that long ago!). We quickly decided that the book would principally target the BizTalk development community and that it would provide a solid and comprehensive introduction to the chief artefacts of BizTalk Server 2010 solutions – schemas, maps, orchestrations, pipelines and adapters. Much of this content was written by Jan Eliasen and forms part 1 (“The Basics”) of the book.

On the day my complimentary copies were delivered, I was working on the implementation of a pipeline component, and had an issue to do with exposing developer-friendly info in Visual Studio. I used this as a test-run of Jan’s content, and sure enough, discovered that he had clearly addressed the issue I had, including sample code. Jan’s contribution is succinct and to the point, but is also very comprehensive (he’s even documented things like creating custom pipeline templates!). I particularly appreciate the way he included plenty of guidance on testing individual artefacts.

My contributions to part 1 is a chapter on adapters (the ‘adapter chapter’ as we fondly called it). This explores each of the ‘native’ adapters and the family of WCF adapters. There is also some content on the new SQL adapter which is part of the BizTalk Adapter Pack. In that respect, it overlaps with ‘Microsoft BizTalk 2010 Line of Business Systems Integration’ which I reviewed recently, and also in respect of the SharePoint adapter. However, ‘Microsoft BizTalk 2010 Line of Business Systems Integration’ provides a whole lot more information on a range of LoB adapters. It is written in a different style to BizTalk Server 2010 Unleashed and is highly complementary.

Although the original plan was to include content on custom adapter creation, this didn’t, in the end, get covered in any depth. One reason for this is that, going forward, most custom adapter development for both BizTalk and Azure Integration Services (still some way off) is likely to be done using the WCF LoB Adapter SDK. That suggested that we would have had to document two distinct adapter frameworks in order to do the job properly, and this proved a little too much to tackle. Room there for another book, methinks.

Part 1 accounts for about half the content of the book. Beyond this, we wanted to add value by covering more advanced topics, including the use of BizTalk Server alongside WCF and the emerging Azure platform, new features in BizTalk Server 2010 and topics that have been only partially covered elsewhere. So, for example, Anush Kumar was contributed an entire section (part 4) on RFID including the new RFID Mobile Framework. Anush is well-known in the BizTalk community due to his involvement in the development of RFID Server. Between Jon Flanders and Brian Loesgen, the book includes content on exploiting WCF extensibility in BizTalk, integrating via the Azure service bus (please note that this content was written before the advent of topics/subscriptions or Integration Services), the BAM framework and the ESB toolkit.

There is also a whole section (part 3) written by Scott Colestock that introduces the Administration Console and describes deployment approaches for BizTalk solutions.

Rules

That leaves one more subject for which I was responsible. One of the main reasons I was asked to contribute to the book was to document rules processing. Although there is some great content out there on the use of the BRE, I have long felt there is a need for a more comprehensive introduction. Due to some early confusion, I originally intended a total of seven short chapters on rules, but this content was refactored into two longer chapters. The first chapter introduces the Business Rules Framework. My idea was to emphasise the entire framework up front, rather than simply explore the rules composer and other tools. I also tried to explain the typical ‘feel’ of rules processing in the context of a BizTalk application, and the relationship between executable rules and higher-level business rules.

The second chapter investigates rule-based programming. It attempts broadly to achieve two related goals. The first is to explain rules programming to developers, to demystify the model, explain the techniques and provide insight into how to handle a number of common issues and pitfalls that rules developers face. The second is to provide a solid theoretical introduction to rules processing, including concepts that are not generally familiar to the average developer. I resisted the temptation, though, to provide an in-depth explanation of how the Rete Algorithm works, which I’m sure will be a relief :-) You can read the Wikipedia article on that.

Conclusions

So there you have it. BizTalk Server 2010 is a mature enterprise-level product which, although it has a long future ahead of it, won’t change fundamentally over time. Microsoft has publically stated that their future major investments in EAI/EDI will be made in the Azure space, although new versions of BizTalk Server will continue to benefit from general improvement and greater integration with the evolving Azure platform. So, hopefully, our content will serve for some time as a useful introduction to BizTalk Server, chiefly from a developer’s perspective.

Share This Post:

Posted On Sunday, September 25, 2011 10:30 PM | Feedback (8)

Monday, September 19, 2011 #

Book Review: Microsoft BizTalk 2010 Line of Business Systems Integration

One benefit of my recent experience on a BA flight was that I got plenty of time to read through “Microsoft BizTalk 2010 Line of Business Systems Integration”. I’d promised the publisher weeks ago that I would take a look and publish some comments, but August has been such a busy month for me, and they have had to be patient. I should point out that, for the sake of transparency, that with another BizTalk book about to be released (next week) which I helped co-author, I have an urgent and obvious need to make good on this promise before I start to blog on other stuff.

BTS10LoBI is a really welcome addition to the corpus of BizTalk Server books and fills a conspicuous gap in the market. BizTalk Server offers a wide-ranging library of adapters. The ‘native’ (built-in) adapters understandably get a lot of attention, as do the WCF adapters, but other adapters, such as the LoB adapters and HIS adapters, are often overlooked. I came to the book with the mistaken assumption that its chief focus was on the BizTalk Adapter Pack. This is a pack of adapters built with the WCF-based LoB SDK. In fact, the book follows a much broader path. It is a book about LoB integration in a general sense, and not about one specific suite of adapters. Indeed, it is not simply about adapters. It focuses on integration with various LoB systems, and explains how adapters and other tools are used to achieve this.

This makes for a more interesting read. For example, one, possibly unintended, consequence (given that it represents collaboration between five different authors) is that it illustrates very effectively the spectrum of approaches and techniques that end up being employed in real-world integration. In some cases developers use adapters that offer focused support for metadata harvesting and other features, exploited through tools such as the ‘Consume Adapter Service’ UI. In other cases, they use native adapters with hand-crafted schemas, or they create façade services. The book covers additional scenarios where third-party LoB tools and cloud services (specifically SalesForce) are used in conjunction with BizTalk Server. Coupled with lots of practical examples, the book serves to provide insight into the ‘feel’ of real-world integration which is so often a messy and multi-faceted experience.

The book does not cover the BizTalk Adapter Pack comprehensively. There is no chapter on the Oracle adapters (not a significant issue because they are very similar to the SQL Server adapter) or the Siebel adapter. On the other hand, it provides two chapters on the SAP adapter looking at both IDOC and RFC/BAPI approaches. I particularly welcome the inclusion of chapters on integration with both Dynamics CRM 2011 and Dynamics AX 2009. I learned a lot about Dynamics CRM which I haven’t had occasion personally to integrate with in its latest version. The chapter on SalesForce mentions, but does not describe in any detail, the TwoConnect SalesForce adapter which we have used very effectively on previous projects. Rather, it concentrates on direct HTTP/SOAP interaction with SalesForce.com and, very usefully, advocates the use of Azure AppFabric for secure exchange of data across the internet.

The book provides two chapters on integration with SharePoint 2010. The first explores the use of the native adapter to communicate with form and document libraries, and provides illustrated examples of working with InfoPath forms. It would have been reasonable to stop there, but instead, the second chapter goes on to describe how to integrate more fulsomely with SharePoint via its web service interface, and specifically how to interact with SharePoint lists.

Increasingly, the BizTalk community is waking up to the implications of Windows Azure and AppFabric. This is an important step for developers to take. Future versions of BizTalk Server will essentially join and extend the on-premise AppFabric world. As Microsoft progressively melds their on/off premise worlds, BizTalk developers will increasingly have to grapple with integration of cloud based services, and integration of on-premise services via the cloud. The book is careful to address this emerging field through the inclusion of a chapter on integration via the Azure AppFabric service bus. As I mentioned above, this is applied specifically to SalesForce integration in a later chapter. The AppFabric Service Bus is a rapidly-evolving part of the Azure platform, and is set to introduce a raft on new features in the coming months which will greatly extend the possibilities. Eventually we will see cloud-based integration services appear in this space. So, the inclusion of this chapter points out the direction of major future evolution of Microsoft’s capabilities and offerings in the integration space.

The book is not shy about providing guidance on practical problems and potential areas of confusion that developers may encounter. The content is clearly based on real-world experience and benefits from ‘war stories’. The value of such content cannot be underestimated, and can save developers hours of pain and frustration when tackling new problems. All in all, I thoroughly welcome this book. My thanks to the authors, Kent Waere, Richard Seroter, Sergei Moukhnitski, Thiago Almeida and Carl Darski.

Share This Post:

Posted On Monday, September 19, 2011 10:47 PM | Feedback (0)

Sunday, September 18, 2011 #

Broken British Airways

I'm sitting is a nice new hotel in Redmond - the Hotel Sierra is well worth considering if you are staying in the area. I'm sleep-deprived and jet-lagged, and it's raining hard outside, but hey, I just got to play with one of the Samsung tablets they handed out at Build, and was not disappointed. Microsoft is doing something trully remarkable with Win8 Metro.

On the other hand, I am deeply disappointed with the UK flag carrier, British Airways. Indeed, I've lost patience with them big-time. So forgive me for getting this off my chest. I am very much in the mood to do as much reputational damage to them as I can.

When I checked in on-line, they had booked me into one seat but I could see another with more legroom (a front row). Because of repeated experience over the last few years with defective headsets (I always carry my own earphones these days after one flight here we went through three different headsets before finding one in which one of the earphones actually worked) and bad headset connections (having to constantly twiddle the jack to try to hear anything), I spent a little while consciously debating with myself the intangible risks of changing my seat – i.e., I could easily be swapping a ‘working’ seat for a broken ‘one’. Of course, there was no way to know, so I opted for the seat with more legroom.

MISTAKE! Forget about dodgy headsets. Nothing worked. Not even the reading light! Certainly not the inflight entertainment. They failed to show me the safety video (the steward did panic a little when he realised they had failed to comply with their legal obligations). So I sat for 9.5 hours in a grubby, worn-out cabin with nothing!

To be fair, they did offer to try to find me another seat (the plane was very full), but I opted for the legroom because I wanted to try to get some sleep. So I could probably have got in-flight entertainment. The point is, though, that this is now more than just an unfortunate couple of co-incidences over the last two or three years. I am reasonably fair-minded and understand that sometimes, with the best will in the world, things just go wrong. In any case, I was bought up to put-up or shut-up (as my mother would say - it's part of the culture). However, I am forced to conclude that this is now a repeated trend that I experience regularly to the point where I am consciously suspicious of the seats they give me, and clearly with good reason. BA simply fails to maintain its cabins to anything like a reasonable or acceptable standard (I must trust they do a better job in maintaining the engines). I used to feel some patriotic pride in BA. Not now. It’s so sad to see the British flag carrier consistently deliver such an embarrasingly poor and second-rate service. I will be asking SolidSoft in future to, where possible, book me onto a different carrier and will do what I can to convince the company to use other carriers by default.

Personally, I think the UK government should give flag carrier status to someone else (Virgin, I guess).

Share This Post:

Posted On Sunday, September 18, 2011 3:04 PM | Feedback (2)

Thursday, September 15, 2011 #

Windows 8, Metro and IE10: First impressions

I've just installed the Windows 8 Developer Preview. These are some first impressions:

Installation of the preview was quite smooth and didn't take too long. It took a few minutes to extract the files onto a virtual image, but feature installation then seemed to happen almost instantaneously (according to the feedback on the screen). The installation routine then went into a preparation cycle that took two or three minutes. Then the virtual machine rebooted and after a couple of minutes more preparation, up came the licence terms page.

Having agreed to the licence, I was immediately taken into a racing-green slidy-slidy set of screens that asked me to personalize the installation, including entering my email address. I entered my work alias. I was then asked for another alias and password for access to Windows Live services and other stuff. There was a useful link for signing up for a Windows Live ID. I duly entered the information. Only on the next screen did I spot an option to not sign in with a Live ID. I didn't try this, but I felt a bit peeved that the use of a Live ID had appeared mandatory until that point. I suspect the idea is to try to entice users to get a Live ID, even if they don't really want one.

A couple more minutes of waiting, et voilà. The Metro Start screen appeared, covered in an array of tiles. Simultaneously I got an email (on my work alias) saying that a trusted PC had been added to my Live account. I clicked the confirmation link, signed into Windows Live and checked that my PC had indeed been confirmed. Then Alan started chatting, but that is a different matter.

Of course, Oracle's Virtual Box (and my Dell notebook) haven't quite mastered the art of touch yet. For non-touch users a scroll bar appears at the bottom of the Metro UI. I had a moment's innocent fun pretending to swipe the screen with my finger while actually scrolling with the mouse. Ah, happy days. Then I discovered that the scroll wheel on my mouse does the equivalent of finger swiping on the Start page.

I opened up IE10. Wow! I thought IE9's minimal chrome story was amazing. IE10 shows how far short IE9 falls. There is no chrome. Nothing. Nadda. Of sure, there is an address box and some buttons. They appear when needed (a right mouse click without touch) and disappear again as quickly as possible. It’s the same with tabs which have morphed, in the Metro UI, into a strip of thumbnails that appear on demand and then get out of the way once you have made your selection. Click on a new tab and you can navigate to a new page or select a page from a list of recents/favourites. You can also pin sites to 'Start', which in this case means that they appear as additional tiles on the Start screen. I played for a minute and then I suddenly experienced the same rush of endorphins that hit me the first time I opened Google Chrome a few years back. Yes, sad to say, I fell in love with a browser! A near invisible browser. A browser that is IE for goodness sake! A browser that does what so many wished IE would do years ago. It gets out of your way.

Do you like traditional tabs? That's not a problem, because the good-ole desktop is just a click (or maybe a tap or a swipe) away. There is even a useful widget on the now-you-see-me/now-you-don't address bar that takes you to desktop view. It is a bit of a one way trip, and results in a new IE frame opening on the desktop for the current page. On the desktop, IE10 looks just like IE9. It is, however, significantly more accomplished, and has closed much of the remaining gap between IE9, the full HTML5 spec and some of the additional specifications that people incorrectly term 'HTML 5'. Microsoft has more than doubled its score on the (slightly idiosyncratic) HTML5 Test site (http://html5test.com/) and now just pips Opera 11.51, Safari 5.1 and Firefox 6 to the post for HTML5 compliance (it beats Firefox by just 2 points, although it is 1 point behind if you take bonus points into consideration) by that measure, although it still falls behind Google Chrome 13.

Pinning caused me some issues which I suspect are simply bugs in the preview. Having pinned a site, every time I went into the Metro version of IE10, I found that I couldn't click on links, hide the address bar, view tabs, etc. I eventually had to kill my IE10 processes to get things working properly again. I noticed that desktop and Metro IE10 processes appear with slightly different icons in the radically redesigned task manager.

One slight mystery here is that the beta of 64-bit Flash worked fine in Desktop view but not in Metro. No doubt this will long since have become a matter of history by the time all this stuff ships.

For a few minutes, I was rather confused about the apparent lack of a proper Start menu in the desktop view. If you click on Start, you go back to the Metro Start page. And then the obvious dawned on me. In effect, the new Metro Start screen is simply an elaboration of the old Start menu. In previous version, when you click Start, the menu pops up on top of the desktop. It is quite rich in previous versions, and allows you to start applications, perform searches for applications and files or undertake various management and administrative tasks. Windows 8 is really not very different. However, the Start menu has now morphed into the new Metro Start page which takes up the whole screen. Instead of a list of pinned and recent applications, the Start screen displays tiles. Move the mouse down to the bottom right corner (I don't know what the equivalent touch gesture is), and up pops a mini Start menu. Clicking 'Start' takes you back to the desktop. Click on 'Search' to search for applications files or settings. The settings feature is really powerful. In fact, in Windows 7, searching for likely terms like 'Display' or 'Network' also returns results for settings, but you get far more hits in Windows 8. The effect is rather like 'God Mode' in Windows 7. [update: no, I'm wrong. Windows 7 gives you a similar number of hits, BUT you need to click the relevant section in the search results to see them all. I've clearly not being using Search effectively to date!]

The mini Start menu is available in the desktop as well. In this case, if you click 'Search', the search panel opens up on the right of the screen and results then open up to take over the rest of the screen. As I experimented, I found that while things were fairly intuitive, the preview does not always work in a totally predictable fashion. I also suspect that the experience is currently better for touch screens than for traditional mice (I note Microsoft is busy re-inventing the mouse for a Windows 8 world - see http://www.microsoft.com/hardware/en-us/products/touch-mouse/microsite/). This is hardly surprising given that Windows 8 is clearly in an early state and is unfinished. I suspect the emphasis to date has been on touch, and not on mouse-driven equivalents.

Once I grasped the essential nature of the Metro Start page and its correspondence to the Start menu is earlier versions of Windows, I began to feel far more comfortable about the changes. Sure, all the marketing hype is about the radical new UI design features. However, this really is just the next stage of the evolution of the familiar Windows UI. Metro is absolutely fabulous as a tablet UI (better than iOS/Android IMHO, which after all, are really just the old 'icons on a desktop' approach with added gestures), and I think it will actually be quite good for desktops, once it is complete. I note, though, that people have already discovered the registry hack to switch Metro off (see http://www.mstechpages.com/2011/09/14/disable-metro-in-windows-8-developer-preview/), and I think MS would be wise to offer this as a proper setting in the release version. I anticipate, though, that I will not be switching Metro off, even on a non-touch desktop.

Shutting down presented a little difficulty. I am used to using the Start menu to do this (the classic 'Start' to stop conundrum in Windows). I couldn't find a 'Shut Down' command on the Start screen. I eventually did Ctrl-Alt-Delete (or rather, Home-Del in Oracle Virtual Box) and then found a Shut Down option at the bottom left of the screen.

Booting the VBox image takes 20 seconds on my machine. 20 seconds! I'll say that again. 20 seconds!!!! Yes, 20 seconds, just about exactly. That's on a virtual machine on my notebook. On the host, it would be significantly faster. This is Windows like we have never known it before. Frankly, it is the ability to boot fast and run Windows happily on ARM devices (I'll have to take that on trust as I haven't yet seen it for real) that are the really important changes. Almost more important than the Metro UI. The nay-sayers and trolls say it can't be done. I think Microsoft has done it, though.

My last foray into Windows 8 this evening was to launch Visual Studio 2011 Express and have a quick peek at the templates for Win8 development. I have a lot to explore.

The say first impressions are the most important. When I saw the on-line video of Windows 8 a couple of months back, I almost fell off my chair in surprise. Now I have got my hands on an early version I am really quite impressed. Like everyone else, I couldn't see how Microsoft could possibly compete against Apple and Google in the tablet space. Now...well...I look forward to seeing if and how Apple and Google will respond. If it is true, as Steve Ballmer states, that Microsoft had 500 thousand downloads of the preview in less than 24 hours, then tectonic plates have already shifted and Microsoft is firmly on track to become a major contender in the tablet space. OK, that's only one in every 14,000 people on the face of planet earth, and yes, the release version of Lion had double that number of hits in the first 24 hours. Nevertheless, it is a huge figure for an early technical preview of an operating system that won't ship for another year. It means people are very, very keen to start developing for Metro (I know we are at SolidSoft). And if Windows 8 succeeds on tablets, what will that mean for Windows Phone which also uses the Metro concept? Don't ever, ever underestimate Redmond.

Share This Post:

Posted On Thursday, September 15, 2011 12:56 AM | Feedback (12)

Wednesday, September 14, 2011 #

BizTalk Server 2010: Loading properties in custom pipeline components

Following the previous post, here is a second bit of wisdom. In the Load method of a custom pipeline component, only assign values retrieved from the property bag to your custom properties if the retrieved value is not null. Do not assign any value to a custom property if the retrieved value is null.

This is important because of the way in which pipeline property values are loaded at run time. If you assign one or more property values via the Admin Console (e.g., on a pipeline in a Receive Location), BizTalk will call the Load method twice - once to load the values assigned in the pipeline editor at design time and a second time to overlay these values with values captured via the admin console. Let's say you assign a value to custom property A at design time, but not to custom property B. After deploying your application, the admin console will display property A's value in the Configure Pipeline dialog box. Note that it will be displayed in normal text. If you enter a value for Property B, it will be displayed in bold text. Here is the important bit. At runtime, during the second invocation of the Load method, BizTalk will only retrieve bold text values (values entered directly in the admin console). Other values are will not be retrieved. Instead, the property bag returns null values. Hence, if your Load method responds to a null by assigning some other value to the property (e.g., an empty string), you will override the correct value and bad things will happen.

The following code is bad:

object retrievedPropertyVal;
propertyBag.Read("MyProperty", out retrievedPropertyVal, 0);

    if (retrievedPropertyVal != null)
    {
        myProperty = (string)retrievedPropertyVal;
    }
    else
    {
        myProperty = string.Empty;
    }

Remove the 'else' block to comply with the inner logic of BizTalk's approach.

Share This Post:

Posted On Wednesday, September 14, 2011 2:48 PM | Feedback (0)

Biztalk Server 2010: Pipeline component fails with "Value does not fall within the expected range."

Here is a small snippet of BizTalk Server wisdom which I will post for posterity. Say you are creating a custom pipeline component with custom properties. You create private fields and a public properties and write all the code to load and save corresponding property bag values from and too your properties. At some point, when you deploy the BizTalk application and test it, you get an exception from within your pipeline stating, unhelpfully, that "Value does not fall within the expected range." Or maybe, while using the Visual Studio IDE, you notice that values you type into custom properties in the Property List are lost when you reload the pipeline editor.

What is going on? Well, the issue is probably due to having failed to initialise your custom property fields. If they are reference types and have a null value, the PipelineOM PropertyBag class will throw an exception when reading property values. The Read method can distinguish between nulls and, say, empty strings, due to the way data is serialised to XML (e.g., in the BTP file). Here is a property initialised to an empty string:

Here is the same property set to null:

The first is OK. The second causes an error and leads to the symptoms described above.

ALLWAYS initialise property backing fields in custom pipeline components. NEVER set properties to null programmatically.

Share This Post:

Posted On Wednesday, September 14, 2011 1:36 PM | Feedback (0)

Monday, August 22, 2011 #

Andrew Ng to speak at Rules Fest

In my previous post I mentioned the free AI course being run by Peter Norvig and Sebastian Thrun (122,314 and rising) in conjunction with Stanford University School of Engineering. Professor Andrew Ng is running a related course on Machine Learning. This is also a free on-line course run along the same lines as the AI course. Over 30,000 people have signed up so far.

I mention this because Andrew has just confirmed that he will be speaking at this year’s Rules Fest. Rules Fest is all about the practical application by developers of reasoning technologies to real-world problems. It brings together people from across the whole spectrum of public and private sector organisations, including commercial and research organisations and academia, to inspire, inform and enlighten developers and architects. Machine learning is central to the rapidly evolving world of intelligent systems, and we are very excited that Andrew will be speaking at the event.

Share This Post:

Posted On Monday, August 22, 2011 11:41 PM | Feedback (0)

Saturday, August 20, 2011 #

Huge response to on-line AI course

Peter Norvig and Sebastian Thrun are offering a free on-line course on AI later this year in conjunction with Stanford University. The course is broadly based on Peter Norvig's book "Artificial Intelligence: A modern Approach" written jointly with Stuart Russell. Along with my colleagues on the Rules Fest committee, we have been following this with interest. In a few days, well over 100,000 people have signed up (112,774 at the time of writing, and still increasing fast). The course broadly overlaps with our natural areas of interest at Rules Fest which is all about the practical application of reasoning technologies in real-world computing. It is very encouraging to us to see the huge interest this course is generating. We will doubtless be contacting Peter, yet again, to see if he will speak at next year's conference (we keep plugging away at this).

In another development, we all woke up to the news a couple of days ago that HP, as part of its dramatic change in strategy, has bid almost $11Bn to acquire enterprise search company, Autonomy. Autonomy offers proprietary technology that exploits Bayes theorem, Shannon's information theory and specific forms of SVD to create an intelligent search platform with learning capabilities. Clearly, HP sees this type of technology as playing a major and lucrative role in their future.

Some time ago, at an event organised by the excellent BizTalk Users' Group in Sweden, I was asked to do a little crystal ball gazing. I trotted out the line that the next few years will see AI-related and reasoning technologies, formally thought of as esoteric and impractical, find their place at the heart of enterprise computing alongside existing investments in traditional LoB/Back Office applications and integration services. With the advent of cloud computing and platforms such as Azure, we have the horsepower available to make this a practical and feasible possibility for mainstream enterprise computing. AI used to be a dirty word. No longer!

Share This Post:

Posted On Saturday, August 20, 2011 10:13 PM | Feedback (0)

Tuesday, June 21, 2011 #

Windows Azure AppFabric SDK - June CTP - Download issues

Microsoft has announced availability of the June CTP for Windows Azure AppFabric. See http://blogs.msdn.com/b/appfabric/archive/2011/06/20/announcing-the-windows-azure-appfabric-june-ctp.aspx. This is an exciting release and provides greater insight into where the AppFabric team is heading in terms of developer and management tooling. Microsoft is offering space in the cloud to experiment with the CTP, but this is limited, so register early to get a namespace!

You can download the SDK for the June CTP. However, we ran into a lot of trouble trying to do this today. Whenever we followed the link, we ended up on the page for the May CTP. We found what appeared to be a workaround which we were able to repeat on another box (and which I reported on Connect), but then a few minutes later I couldn't repeat it. Just now, the given link appears to be working every time in IE, but not in Firefox! Frankly, the behaviour seems random! It looks like the same URL points to two different pages, and I suspect that which page you end up on is hit and miss.

The link to the download page is http://www.microsoft.com/download/en/details.aspx?id=17691. If you end up on the wrong page, try again later and you may get to the right place. Or try googling "Windows Azure AppFabric SDK CTP – June Update" and following a link to this page. For some reason, that sometimes seems to work.

Good luck!

Share This Post:

Posted On Tuesday, June 21, 2011 5:06 PM | Feedback (0)

Thursday, June 02, 2011 #

Windows Azure AppFabric May and June CTPs - A summary

I spent some time today summarising the new features in the Windows Azure AppFabric May CTP for SolidSoft consultants. Microsoft released the CTP a couple of weeks ago and has a second CTP coming out later this month. I might as well publish this here, although it has been widely blogged on already. There is nothing that you can’t glean from reading the release documents, but hopefully it will serve as a shorter summary.

The May CTP is all about the AppFabric Service Bus. The bus has been extended to support ‘Messaging’ using ‘Queues’ and ‘Topics’

‘Queues’ are really the Durable Message Buffers previewed in earlier CTPs. MS has renamed them in this CTP. They are not to be confused with Queues in Windows Azure storage! Think of these as ‘service bus queues’. They support arbitrary content types, rich message properties, correlation and message grouping. They do not expire (unlike in-memory message buffers). They allow user-defined TTLs. Queues are backed by SQL Azure. Messages can be up to 256KB and each buffer has a maximum size of 100 MB (this will be increased to at least 1GB in the release version). To handle messages larger than 256KB, you ‘chunk’ them within a session (rather like BTS large message handling for MSMQ). The CTP currently limits you to 10 queues per service namespace.

Service Bus queues are quite similar to Azure Queues. They support a RESTful API and a .NET API with a slightly different set of verbs – Send (rather than Put), Read and Delete (rather than Get), Peek-Lock (rather than ‘Peek’) and two verbs to act on locked messages – Unlock and Delete. The locking feature is all about implementing reliable messaging patterns while avoiding the use of 2-phase-commit (no DTC!). Queue management is very similar, but configuration is done slightly differently. AppFabric provides dead letter queues and message deferral. The deferral feature is a built-in temporary message store that allows you to resolve out-of-order message sequences. Hey, this stuff is actually beginning to get my attention!

Today’s in-memory message buffers will be retained for the time being. MS is looking at how much advantage they provide as low-latency non-resilient queues before making a decision on their long-term future. This is beginning to sound like the BizTalk Server low-latency debate all over again! Currently, the documented recommendation is that we migrate to queues.

‘Topics’ provide new pub/sub capabilities. A topic is…drum roll please…a queue! The main difference is that it supports subscription. I assume it has the same limitations and capabilities as a normal queue, although I haven’t seen this stated. It is certainly built on the same foundation. You can have up to 2000 subscriptions to any one topic and use them to fan messages out. Subscriptions are defined as simple rules that are evaluated against user and system-define properties of each message. They have a separate identity to topics. A single subscription can feed messages to a single consumer or can be shared between multiple consumers. Unlike Send Port Groups in BizTalk, this multi-consumer model supports an ‘anycast’ model for competing consumers where a single consumer gets a message on a first-come-first-served basis. MS invites us to think of a subscription as a ‘virtual queue’ on top of the actual topic queue. Potential uses for anycasting include basic forms of load balancing and improved resilience.

The CTP supports AppFabric Access Control v2.0. It is fully backward-compatible with the current service bus capabilities in AppFabric.

CTP does not have load balancing and traffic optimization for relay. These were in earlier CTPs, but have been removed for the time being. They may reappear in the future.

June CTP

The June CTP will introduce CAS (Composite Application Services). CAS is a term used by other vendors (e.g., SAP) for similar features, and has been a long time coming in the Microsoft world. The basic idea is that you build a model of a composite application, the services it contains, its configuration, etc., and then drive a number of tasks from this model such as build and deployment, runtime management and monitoring. Some of us remember an ancient Channel 9 video on a BizTalk-specific CAS-like modelling facility that MS were working on years ago. It was entirely BizTalk-specific and never saw the light of day. However, one connection to make is that CAS will provide capabilities that are conceptually related to the notion of ‘applications’ in BizTalk Server.

We will get a graphical Visual Studio modelling tool to design and manage CAS models. The CAS metamodel is implemented as a .NET library, allowing models to be constructed programmatically. Models are consumed by the AppFabric Application Manager in order to automate deployment, configuration, management and monitoring of composite applications.

So, things are rapidly evolving. However, we won’t see anything on Integration Services until, I suspect, next year. It’s important to remember that the May CTP is all about broadening the existing Service Bus with messaging capabilities, rather than about delivering an integration capability. So, even though we are seeing more BizTalk Server-like features, we are still a long way off having what Burley Kawasaki called a “true integration service” in the cloud. Obviously, Azure Integration Services will exploit and build on the Service Bus, but a lot more needs to be done before we have integration-as-a-service as part of the Azure story.

Share This Post:

Posted On Thursday, June 02, 2011 1:21 PM | Feedback (0)

Wednesday, April 13, 2011 #

IE10 Preview - How Microsoft Competes in the Feature Olympics

Internet Explorer has made huge strides in the last couple of years. Microsoft has at last begun to lay the ghost of IE6 to rest with a solid, fast, standards compliant, reasonably up-to-date (not quite the same thing) forward-looking browser with the cleanest UI in the business.

However, one issue clouds the horizon. Other browsers, most notably Google Chrome, rev much faster that IE. The importance of this is that, by bringing out new versions on shorter (much shorter, in some cases) timescales, other browsers gain a clear advantage. They get emerging specifications and technologies out to end-users faster and therefore shape the future of the web in a way that IE could not touch for many years. This, in turn, builds a sense of momentum which increases the loyalty of the user base. Indeed, it is a major factor in growing the user-base, as we can see clearly with Chrome. With IE, by contrast, we have lived for years with a strong sense of Microsoft holding everyone else back.

Microsoft needs to speed up the cycle. I wondered if we would see any signs of this after the RTW of IE9 last month. Well, yes we can. Yesterday, Microsoft released Platform Preview 1 of IE10. Given Microsoft's record, this is absolutely astounding and provides further evidence that the bad old days for IE really are receding. Of course, after three weeks of development, things are not that different to IE9, but the HTML5 story is improved, thanks chiefly to several new CSS3 features.

IE9's HTML5 story is mixed. On the one hand, the browser lays a strong foundation for the future with comprehensive DirectX rendering and a much-repeated commitment to HTML5 as a future standard. On the other hand, the feature list count for HTML5 is noticeably lower than its nearest rivals. Put simply, IE9 supports less HTML5 than others, but supports it well. The IE team has repeatedly stated its position on this. Their main argument is that IE9 supports 'site-ready' HTML5 that will interoperate across all today's browsers and avoids features on which there is still disagreement or which may not make it to the final specification. A secondary theme is that it was more important to develop the sub-systems that underpin a great HTML5 experience than strive after having the longest HTML5 feature list.

Broadly, the argument plays quite well. Given IE's poor reputation amongst web developers, it is understandable that they want to stress their commitment to interoperability. Certainly, some of the other browsers are now carrying the burden of 'HTML5' features that won't actually make it into the W3C Recommendation (whenever it finally arrives) and features which are not broadly interoperable with other browsers. Of course, it's not really as clear cut as this. Some of the features that were once counted as part of HTML5 are still very much alive and kicking and will almost certainly be incorporated into future versions of HTML.

Try using each of the HTML5-enabled browsers in turn to browse the various HTML5 showcases provided by different vendors, and you will quickly get an idea of where HTML5 interoperability is really at, currently. And yes, the other HTML5 browsers tend to fare well on Microsoft's HTML5 preview site in terms of functionality, suggesting that Microsoft's take on HTML5 does indeed approximate to a 'lowest common denominator' interoperable approach. Of course, performance is a different story and seems to be the dominant theme of Microsoft's showcase apps.

The myth that Microsoft spends its days subverting every standard going whilst all other players effortlessly deliver interoperable perfection has never been accurate since a brief period in the first half of the 1990's when the company first ‘won’ this reputation thanks to some fairly cynical actions (nothing, of course, to do with web standards). It is certainly the case that the complaint of poor standards compliance can no longer be honestly sustained with respect to IE9. However, IE9 illustrates a dilemma that all browser vendors face. It can take an eon for emerging web specifications to stabilise and for standards organisations to ratify and publish a given standard. It simply isn’t the role of organisations like the W3C to be in the vanguard of developing new specifications for everyone else to follow. There has been a huge misunderstanding about this for years. The W3C, and others, follow where the industry leads. They work to foster collaboration and agreement amongst those who are actually coming up with the ideas, trying out new innovations and pushing the boundaries. That means that the more innovative browsers will always be ahead of the curve and, consequently, in danger of subverting interoperability.

This, then, is the big question about IE’s HTML5 support. Now that Microsoft has convincingly closed the field on its rivals, will it be content simply to follow where others lead, trailing at the rear end of a densely-packed group of HTML.next ‘feature-athletes’? Or should it attempt to move closer to the front of the pack in order to more effectively set the pace. Well, long-distance running is all about pacing and strategy. If you are in the race for the long-haul, it often pays to hold back for a while. I think it probably suits Microsoft's purposes for the time being to merely match the pace set by others whilst all the time quietly building on their blindingly-fast DirectX rendering engine. And after all, we are still some way off seeing an equivalent Microsoft HTML5 technology for mobile devices which is where, increasingly, it really counts. I suspect they are attempting to maintain their stamina and build up their strength for a future push to the front. After all, Microsoft always preferred being in the industry driving seat.

Share This Post:

Posted On Wednesday, April 13, 2011 9:09 PM | Feedback (0)

Tuesday, April 12, 2011 #

UML Use Cases - The Tomb of the Unknown Semantic

A business analyst colleague asked me to comment on how he could improve a use case diagram he had inherited. Someone had carefully transposed the outcome of a whiteboard session into a Visio UML diagram. We discussed the difference between <<includes>> and <<extends>> and I whinged on about how 'traditional' UML use case diagrams don't always convey very useful information. I made one practical suggestion, though. The use cases and associations were all neatly colour-coded using two colours (red and green). However, I couldn't tell what these colours signified, so I suggested that he annotate the Visio diagram to make the semantics clear.

It turns out that the colours reflect the fact they had two different whiteboard pens.

Aaaaaaagh.

Tomb of the Unknown Semantic

Share This Post:

Posted On Tuesday, April 12, 2011 12:17 PM | Feedback (0)

Wednesday, April 06, 2011 #

Rules Fest 2011 - Bigger, Bolder, Better

Well, it's official. Rules Fest 2011 is on track for October. The web site has been launched and the "Call for Speakers" has been published. Welcome to the world’s premier technical conference for reasoning technologies.

http://rulesfest.org/html/home.html

Building on the success of last year's conference, this year will be bigger and better in every way. We’ve hired a larger venue - the Hyatt Regency in Burlingame, just south of SFO. We really loved being at the Dolce Hayes last year (warmly recommended), and it was sad to have to move on, but we need more space to accommodate a significantly larger event with exhibitions, a career centre and much more. We are also running an extensive program of boot camp sessions. The hotel has great amenities and is conveniently located for Silicon Valley and excursions to San Francisco.

We have two keynote speakers this year. Paul Haley is one of the best-known names in the industry – a co-developer of ART and ex-chairman of Haley Systems, subsequently acquired by Oracle. His company was responsible for the Haley Business Rules Suite (now evolved into Oracle Policy Automation). Paul is a consultant at Paul Allen’s company, Vulcan, where he is involved with Project Halo. Said Tabet, another ART veteran, co-founded the RuleML Initiative a decade ago. He has contributed extensively to the development of W3C and OMG rules standards over several years and is focussed on the Semantic Web and the use of reasoning technologies in financial services.

As vice-chair, my job is to oversee the call for speakers and plan the detailed agenda over the next couple of months. If you work with rules engines or other reasoning technologies and have a burning desire to share your knowledge, experience and insight with developers, technologists, researchers, technical leads, etc., then submit a short abstract for a presentation no later than 6^th May. You’ll find all the details and T&Cs on the web site. If you just need to learn more about the practical application of reasoning systems (rules engines, constraint solvers and the like) to real-world problems, then book yourself a place at the conference. Book by May 15^th to get the best discounts.

With Microsoft working on revamped rules offerings for AppFabric (as mentioned at last year's PDC), it would be great to see a good .NET turnout at the event. Let me know if you plan to attend and are a .NET developer with a passion for rules. If you are a BizTalk developer, even better. I would love to do a boot camp for MS BRE, but would need some indication of committment to this a.s.a.p. in order to look for the funding to host a day of intensive hands-on experience. So, if you are interested, ping me via the 'Contact' menu option on this site.

Share This Post:

Posted On Wednesday, April 06, 2011 1:03 AM | Feedback (0)

Monday, April 04, 2011 #

...and the JavaScript winner is....

We all love benchmarks! With the recent release of new versions of some of major browsers, and as a small diversion over the weekend, I ran five well-known browsers against five well-known JavaScript micro-benchmarking suites using my laptop. The results are reproduced below. I have ranked the results for each benchmark suite from best to worst.

Celtic Kane – old version (current version was unavailable)
(Smaller is better)

Opera 11		77 ms
Safari 5		93 ms
Chrome 10		119 ms
IE 9		152 ms
FF 4		154 ms

Kraken 1.0 (Mozilla)
(Smaller is better)

FF 4		7,555.6 ms
Chrome 10		8,439.8 ms
Opera 11		12,918.8 ms
IE 9		16,551.9 ms
Safari 5		19,099.8 ms

Dromaeo (Mozilla) (all JavaScript tests)
(Bigger is better)

Chrome 10		457.53 runs/s
IE 9		403.96 runs/s
FF 4		386.74 runs/s
Opera 11		369.49 runs/s
Safari 5		257.42 runs/s

V8 v6 (Google)
(Bigger is better)

Chrome 10		7,737
FF 4		3,111
Opera 11		3,050
Safari 5		2,319
IE 9		2,119

SunSpider 0.9.1 (WebKit)
(Smaller is better)

IE 9		249.8 ms
Opera 11		289.9 ms
FF 4		295.2 ms
Chrome 10		309.0 ms
Safari 5		353.9 ms

So, what does this prove? Absolutely nothing! It's impossible to pick an overall winner from these results, or even to determine any particular trend. I'll provide two observations, however. First, comparative micro-benchmarking remains as problematic as ever. Pick your preferred test to 'prove' whatever you wish. Second, competition between browsers remains fierce. As a result, JavaScript performance has improved massively across the board in the last couple of years. That's great news. It means we are all winners!

Share This Post:

Posted On Monday, April 04, 2011 9:44 PM | Feedback (0)

Don't hire bad developers - period.

There has been quite a furore in the last couple of weeks about a blog article published by David Barrett entitled "CEO Friday: Why we don’t hire .NET programmers". You can find it at http://blog.expensify.com/2011/03/25/ceo-friday-why-we-dont-hire-net-programmers/. Along with about half the global development community, I tried to post a response, but comments were already closed. I had all but forgotten the article until, a few minutes ago, up popped a daily .NET newsletter with a headline linking to the article.

I can't help thinking this is just one step removed from a long line of '.NET developers are dumb' articles. I've read a few of these over the years. The last one I remember (I can't recall the URL, unfortunately) was classic. The guy writing the article explained that he had recently interviewed half a dozen developers for a job. All but one were Java developers, and he was generally impressed by their understanding of design patterns and the like - even a candidate fresh from college. One, however, was a .NET developer (I wonder if the interviewee had misread the job description) and he was apparently very ignorant. On the basis of this deeply representative sample, the author concluded that the whole .NET development community (several million people) are equally useless and ignorant! I kid you not!

Maybe I'm being unfair to David Barrett. However, while posting a comment to his blog site would have made me feel better, it wouldn't have achieved anything. Reading through the comments, I'm struck how many posts quickly degenerate into mud-slinging. David is accused of several things which are not worth repeating. In turn, others rise to 'defend' his position by throwing vitriol and invective at the whole tribe of .NET developers.

For my part, and simply as a self-indulgent exercise, I will note that I do not recognise David's characterisation of .NET development. It seems to me that he fails to explain properly what aspects of .NET tools and technologies he is talking about. In a subsequent update, he explained his original claim that .NET is a 'language' was a poor 'word choice'. Fair enough. However, it seems to me that he describes .NET all the way through his post as if it is some kind of 4GL. It patently isn't! More to the point, neither are the mainstream .NET languages. Maybe he has in mind Microsoft's old love of 1990's-style form development, as originally used in .NET not only for desktop app development but also 'classic' ASP.NET. If this is the case, he is seriously out of date. Professional ASP.NET development has long-since moved foursquare into the MVC / RESTful world. Desktop development (now a minority sport in this web-enabled, mobile age) has largely moved on to XAML-based development. Maybe that's the issue. XAML can be considered to be just one of many forms of model-driven development Maybe he is railing against MDD. If so, I can't imagine why he has singled out .NET. If anything, .NET has been playing a degree of catch-up with the Java world on that score. So, I remain confused by his arguments (no doubt some will claim I'm just too dumb), but confident from my own experience that he is deeply mischaracterising the .NET platform and the community of .NET developers.

Well, I've got that off my chest. A wider issue, though, arises from reading some of those comments. Why, after all these years, are we developers are so ready to sling mud at others. We fall hook, line and sinker for simplistic, ill-informed characterisations of whole swathes of our industry in an attempt to convince ourselves that we are the clever chosen elect and those others over there are just plain stupid and damned to all eternity. It's so childish. It's so false. In summary, my take on David's article, which I consider a tad more balanced, is this. Don't hire bad .NET developers. Don't hire bad Java developers. Don't hire bad Ruby or C++ or Python, or Perl developers. There’s no need. There are plenty of good developers out there. This is a great industry.

Share This Post:

Posted On Monday, April 04, 2011 9:00 PM | Feedback (0)

Wednesday, February 09, 2011 #

RIF PRD: Presentation syntax issues

Over Christmas I got to play a bit with the W3C RIF PRD and came across a few issues which I thought I would record for posterity. Specifically, I was working on a grammar for the presentation syntax using a GLR grammar parser tool (I was using the current CTP of ‘M’ (MGrammer) and Intellipad – I do so hope the MS guys don’t kill off M and Intellipad now they have dropped the other parts of SQL Server Modelling). I realise that the presentation syntax is non-normative and that any issues with it do not therefore compromise the standard. However, presentation syntax is useful in its own right, and it would be great to iron out any issues in a future revision of the standard.

The main issues are actually not to do with the grammar at all, but rather with the ‘running example’ in the RIF PRD recommendation. I started with the code provided in Example 9.1. There are several discrepancies when compared with the EBNF rules documented in the standard. Broadly the problems can be categorised as follows:

· Parenthesis mismatch – the wrong number of parentheses are used in various places. For example, in GoldRule, the RHS of the rule (the ‘Then’) is nested in the LHS (‘the If’). In NewCustomerAndWidgetRule, the RHS is orphaned from the LHS. Together with additional incorrect parenthesis, this leads to orphanage of UnknownStatusRule from the entire Document.

· Invalid use of parenthesis in ‘Forall’ constructs. Parenthesis should not be used to enclose formulae.

Removal of the invalid parenthesis gave me a feeling of inconsistency when comparing formulae in Forall to formulae in If. The use of parenthesis is not actually inconsistent in these two context, but in an If construct it ‘feels’ as if you are enclosing formulae in parenthesis in a LISP-like fashion. In reality, the parenthesis is simply being used to group subordinate syntax elements. The fact that an If construct can contain only a single formula as an immediate child adds to this feeling of inconsistency.

·      Invalid representation of compact URIs (CURIEs) in the context of Frame productions. In several places the URIs are not qualified with a namespace prefix (‘ex1:’). This conflicts with the definition of CURIEs in the RIF Datatypes and Built-Ins 1.0 document. Here are the productions:

CURIE          ::= PNAME_LN
                 | PNAME_NS
PNAME_LN       ::= PNAME_NS PN_LOCAL
PNAME_NS       ::= PN_PREFIX? ':'
PN_LOCAL       ::= ( PN_CHARS_U | [0-9] ) ((PN_CHARS|'.')* PN_CHARS)?
PN_CHARS       ::= PN_CHARS_U

The more I look at CURIEs, the more my head hurts! The RIF specification allows prefixes and colons without local names, which surprised me. However, the CURIE Syntax 1.0 working group note specifically states that this form is supported…and then promptly provides a syntactic definition that seems to preclude it! However, on (much) deeper inspection, it appears that ‘ex1:’ (for example) is allowed, but would really represent a ‘fragment’ of the ‘reference’, rather than a prefix! Ouch! This is so completely ambiguous that it surely calls into question the whole CURIE specification. In any case, RIF does not allow local names without a prefix.

· Missing ‘External’ specifiers for built-in functions and predicates.

The EBNF specification enforces this for terms within frames, but does not appear to enforce (what I believe is) the correct use of External on built-in predicates. In any case, the running example only specifies ‘External’ once on the predicate in UnknownStatusRule. External() is required in several other places.

· The List used on the LHS of UnknownStatusRule is comma-delimited. This is not supported by the EBNF definition. Similarly, the argument list of pred:list-contains is illegally comma-delimited.

· Unnecessary use of conjunction around a single formula in DiscountRule. This is strictly legal in the EBNF, but redundant.

All the above issues concern the presentation syntax used in the running example. There are a few minor issues with the grammar itself. Note that Michael Kiefer stated in his paper “Rule Interchange Format: The Framework” that:

“The presentation syntax of RIF … is an abstract syntax and, as such, it omits certain details that might be important for unambiguous parsing.”

· The grammar cannot differentiate unambiguously between strategies and priorities on groups. A processor is forced to resolve this by detecting the use of IRIs and integers.
This could easily be fixed in the grammar.

· The grammar cannot unambiguously parse the ‘->’ operator in frames. Specifically, ‘-’ characters are allowed in PN_LOCAL names and hence a parser cannot determine if ‘status->’ is (‘status’ ‘->’) or (‘status-’ ‘>’). One way to fix this is to amend the PN_LOCAL production as follows:

PN_LOCAL ::= ( PN_CHARS_U | [0-9] ) ((PN_CHARS|'.')* ((PN_CHARS)-('-')))?

However, unilaterally changing the definition of this production, which is defined in the SPARQL Query Language for RDF specification, makes me uncomfortable.

· I assume that the presentation syntax is case-sensitive. I couldn’t find this stated anywhere in the documentation, but function/predicate names do appear to be documented as being case-sensitive.

· The EBNF does not specify whitespace handling. A couple of productions (RULE and ACTION_BLOCK) are crafted to enforce the use of whitespace. This is not necessary. It seems inconsistent with the rest of the specification and can cause parsing issues. In addition, the Const production exhibits whitespaces issues. The intention may have been to disallow the use of whitespace around ‘^^’, but any direct implementation of the EBNF will probably allow whitespace between ‘^^’ and the SYMSPACE.

Of course, I am being a little nit-picking about all this. On the whole, the EBNF translated very smoothly and directly to ‘M’ (MGrammar) and proved to be fairly complete. I have encountered far worse issues when translating other EBNF specifications into usable grammars. I can’t imagine there would be any difficulty in implementing the same grammar in Antlr, COCO/R, gppg, XText, Bison, etc.

A general observation, which repeats a point made above, is that the use of parenthesis in the presentation syntax can feel inconsistent and un-intuitive. It isn’t actually inconsistent, but I think the presentation syntax could be improved by adopting braces, rather than parenthesis, to delimit subordinate syntax elements in a similar way to so many programming languages. The familiarity of braces would communicate the structure of the syntax more clearly to people like me.

If braces were adopted, parentheses could be retained around ‘var (frame | ‘new()’) constructs in action blocks. This use of parenthesis feels very LISP-like, and I think that this is my issue. It’s as if the presentation syntax represents the deformed love-child of LISP and C. In some places (specifically, action blocks), parenthesis is used in a LISP-like fashion. In other places it is used like braces in C. I find this quite confusing.

Here is a corrected version of the running example (Example 9.1) in compliant presentation syntax:

Document(

Prefix( ex1 <http://example.com/2009/prd2> )

(* ex1:CheckoutRuleset *)

Group rif:forwardChaining (

(* ex1:GoldRule *)

Group 10 (

Forall ?customer such that And(?customer # ex1:Customer

?customer[ex1:status->"Silver"])

(Forall ?shoppingCart such that ?customer[ex1:shoppingCart->?shoppingCart]

(If Exists ?value (And(?shoppingCart[ex1:value->?value]

External(pred:numeric-greater-than-or-equal(?value 2000))))

Then Do(Modify(?customer[ex1:status->"Gold"])))))

(* ex1:DiscountRule *)

Group (

Forall ?customer such that ?customer # ex1:Customer

(If Or( ?customer[ex1:status->"Silver"]

?customer[ex1:status->"Gold"])

Then Do ((?s ?customer[ex1:shoppingCart-> ?s])

(?v ?s[ex1:value->?v])

Modify(?s [ex1:value->External(func:numeric-multiply (?v 0.95))]))))

(* ex1:NewCustomerAndWidgetRule *)

Group (

Forall ?customer such that And(?customer # ex1:Customer

?customer[ex1:status->"New"] )

(If Exists ?shoppingCart ?item

(And(?customer[ex1:shoppingCart->?shoppingCart]

?shoppingCart[ex1:containsItem->?item]

?item # ex1:Widget ) )

Then Do( (?s ?customer[ex1:shoppingCart->?s])

(?val ?s[ex1:value->?val])

(?voucher ?customer[ex1:voucher->?voucher])

Retract(?customer[ex1:voucher->?voucher])

Retract(?voucher)

Modify(?s[ex1:value->External(func:numeric-multiply(?val 0.90))]))))

(* ex1:UnknownStatusRule *)

Group (

Forall ?customer such that ?customer # ex1:Customer

(If Not(Exists ?status

(And(?customer[ex1:status->?status]

External(pred:list-contains(List("New" "Bronze" "Silver" "Gold") ?status)) )))

Then Do( Execute(act:print(External(func:concat("New customer: " ?customer))))

Assert(?customer[ex1:status->"New"]))))

)

I hope that helps someone out there :-)

Share This Post:

Posted On Wednesday, February 09, 2011 9:26 PM | Feedback (0)

Tellago releases a RESTful API for BizTalk Server business rules

Jesus Rodriguez has blogged recently on Tellago Devlabs' release of an open source RESTful API for BizTalk Server Business Rules. This is an excellent addition to the BizTalk ecosystem and I congratulate Tellago on their work.

See http://weblogs.asp.net/gsusx/archive/2011/02/08/tellago-devlabs-a-restful-api-for-biztalk-server-business-rules.aspx

The Microsoft BRE was originally designed to be used as an embedded library in .NET applications. This is reflected in the implementation of the Rules Engine Update (REU) Service which is a TCP/IP service that is hosted by a Windows service running locally on each BizTalk box. The job of the REU is to distribute rules, managed and held in a central database repository, across the various servers in a BizTalk group. The engine is therefore distributed on each box, rather than exploited behind a central rules service.

This model is all very well, but proves quite restrictive in enterprise environments. The problem is that the BRE can only run legally on licensed BizTalk boxes. Increasingly we need to deliver rules capabilities across a more widely distributed environment. For example, in the project I am working on currently, we need to surface decisioning capabilities for use within WF workflow services running under AppFabric on non-BTS boxes. The BRE does not, currently, offer any centralised rule service facilities out of the box, and hence you have to roll your own (and then run your rules services on BTS boxes which has raised a few eyebrows on my current project, as all other WCF services run on a dedicated server farm ).

Tellago's API addresses this by providing a RESTful API for querying the rules repository and executing rule sets against XML passed in the request payload. As Jesus points out in his post, using a RESTful approach hugely increases the reach of BRE-based decisioning, allowing simple invocation from code written in dynamic languages, mobile devices, etc.

We developed our own SOAP-based general-purpose rules service to handle scenarios such as the one we face on my current project. SOAP is arguably better suited to enterprise service bus environments (please don't 'flame' me - I refuse to engage in the RESTFul vs. SOAP war). For example, on my current project we use claims based authorisation across the entire service bus and use WIF and WS-Federation for this purpose. We have extended this to the rules service. I can't release the code for commercial reasons :-( but this approach allows us to legally extend the reach of BRE far beyond the confines of the BizTalk boxes on which it runs and to provide general purpose decisioning capabilities on the bus.

So, well done Tellago. I haven't had a chance to play with the API yet, but am looking forward to doing so.

Share This Post:

Posted On Wednesday, February 09, 2011 3:35 PM | Feedback (2)

Friday, February 04, 2011 #

The Bing Sting - an alternative opinion

I know I'm a bit of an MS fanboy at times, but please, am I missing something here? Microsoft, with permission of users, exploits clickstream data gathered by observing user behaviour. One use for this data is to improve Bing queries. Google equips twenty of its engineers with laptops and installs the widgets required to provide Microsoft with clickstream data. It then gets their engineers to repeatedly (I assume) type in 'synthetic' queries which bring back 'doctored' hits. It asks its engineers to then click these results (think about this!). So, the behaviour of the engineers is observed and the resulting clickstream data goes off to Microsoft. It is processed and 'improves' Bing results accordingly.

What exactly did Microsoft do wrong here?

Google's so-called 'Bing sting' is clearly a very effective attack from a propaganda perspective, but is poor practice from a company that claims to do no evil. Generating and sending clickstream data deliberately so that you can then subsequently claim that your competitor 'copied' that data from you is neither fair nor reasonable, and suggests to me a degree of desperation in the face of real competition. Monopolies are undesirable, whether they are Microsoft monopolies or Google monopolies. Personally, I'm glad Microsoft has technology in place to observe user behaviour (with permission, of course) and improve their search results using such data. I can only assume Google doesn't implement similar capabilities. Sounds to me as if, at least in this respect, Microsoft may offer the better technology.

[UPDATE]...and here a few links that support a similar perspective or at least offer an even-handed appraisal. For the most part, they are clearly written by people better informed than me. Incidently, I hadn't read any of these links before posting - any similarity really is coincidental. I wasn't copying stuff from Google searches :-)

http://googlewatch.eweek.com/content/google_vs_microsoft/microsofts_mehdi_plays_pot_meet_kettle_with_google.html

http://www.techdirt.com/articles/20110201/11022312911/googles-childish-response-to-microsoft-using-google-to-increase-bing-relevance.shtml

http://www.itworld.com/internet/136002/googles-copying-accusation-called-silly

http://www.businessweek.com/managing/content/feb2011/ca2011024_853469.htm?campaign_id=rss_topStories

http://www.ecommercetimes.com/story/71771.html

http://technorati.com/technology/article/googles-bing-sting-bears-fruitbut-is/

http://blog.moertel.com/articles/2011/02/02/the-google-microsoft-squabble-over-bing-results-some-completely-uninformed-speculation

http://willwhim.wordpress.com/2011/02/02/is-bing-cheating-at-search/

Share This Post:

Posted On Friday, February 04, 2011 7:26 PM | Feedback (3)

Thursday, October 14, 2010 #

Coming up for Air: Rules Fest 2010

Jason Morris called it ‘decompression’. After 10 months of planning and three packed days of presentations here in San Jose, Rules Fest 2010 is now in ‘boot camp’ mode. More than half the attendees stayed for today’s sessions, and are currently spread between four boot camps – JBoss Rules, IBM JRules, OpenRules and Jess. My role as MC and catwalk model (don’t ask!) came to an end last night, and I am sitting in my hotel room, shattered, happy, but also with a distinct feeling of the ‘bends’ as I emerge blinking into the warm Californian sunshine.

We are really happy with the way this year’s conference has gone. These are constrained times, economically, and we needed to prove the viability of the event and the ability of new management, and a new owner, to organise and host an international conference that provides real benefit to its attendees and sponsors. The post mortem will begin in earnest in a few days, and knowing our committee, will be a frank, open and critical appraisal of the last few days. However, we hit our targets, substantially grew the event from last year, garnered great feedback, had a really excellent line-up of speakers and all enjoyed the occasion immensely.

Rules Fest will strive to deepen its commitment to provide a technical conference ‘for developers, by developers’. As I tried to explain to one bemused non-Microsoft person yesterday, we aim to be the ‘PDC’ of the rules processing world, and refuse to be just another trade show. Next year, we will evolve the program to provide deeper developer content and more opportunity for rules developers to share their experience and get direct input into the issues they face. We plan to significantly increase the attendance again (we learned so much this year about how to market it, and are confident that we can hit significantly higher targets next year), attract wider sponsorship and broaden the pool of presenters.

My one on-going pain point is the lack of a really good .NET story. Now don’t get me wrong. I love the Java and C/C++ guys/gals to bits, but I really, really want to see the .NET developers who turn up to the event get more for their money. Mark Proctor quizzed me on my interest in the event last night (and immediately provided a litany of answers to his own questions!). I understand why he is a bemused. Where are the .NET rule vendors? Everyone else is there. Where were you, InRule? We tried to get you interested. Where were you, Microsoft (I know Karl would have loved to be there, as he has been in previous years)? And Dan, can’t we get IBM to push ILOG.NET a bit more? I said to Mark that at some point, the .NET rule logjam has to break, and I fully intend to be there when it does. Rules belong to businesses, not to the JRE. .NET developers do rules as well!

Big thanks to the team at the Hayes Mansion. Inga, you are an absolute star! Big thanks to Brenda who helped make the event a success. And big, big thanks to Jason and Brian who never lost faith.

Share This Post:

Posted On Thursday, October 14, 2010 7:23 PM | Feedback (4)

Sunday, September 26, 2010 #

BizTalk Server 2010 Developer Edition - Installation Issue

I installed the new BizTalk Server 2010 Developer Edition, released a couple of days ago, on a Windows Server 2008 R2 virtual image. The installation went very smoothly, but I did notice one problem. Initially I attempted to upgrade the beta version of BizTalk Server. Brian Loesgen reports that this worked OK for him. However, I couldn't do this. The installer reported that upgrade from the Enterprise Edition to the Developer Edition is not supported. I attempted to uninstall the beta, but got an error message which left my installation in an invalid state.

Fair enough. You have to expect this sort of problem when using betas. That’s one good reason for using virtualisation.

I installed the RTM Developer Edition on a fresh image, and as I say, everything went very smoothly. However, I tried re-running the BizTalk installer and, to my dismay, got a rude message saying:

Cannot uninstall or modify the Dev. edition of BTS 2010

Setup has detected that the Enterprise Edition of BizTalk Server 2010 was installed. Upgrading from the Enterprise Edition to the Developer Edition is not supported.

Please uninstall the Enterprise Edition and then run Setup again.

This is simply not true. This was a completely fresh build of Windows Server on which I have only installed the Developer Edition. I have not tried uninstalling BizTalk to see what happens.

So, be careful with the Developer Edition. My experience is that there is a problem with the installer.

[UPDATE 28/09/2010] Rahul, from MS, is looking into the issue. Meanwhile, early indications are that others are not experiencing the same problem. Thiago Almeida kindly experimented, and was able re-run setup. He is using Windows 7 Enterprise whereas I am using Windows Server 2008 R2. Also, he built a 32-bit installation using WinVPC wheras I built a 64-bit installation using VirtualBox (WinVPC can't run 64-bit images). Meanwhile, over on Connect, Johan Hedberg, reports that things worked OK for him on a 64-bit Windows 7 machine. He uninstalled the beta and then installed the RTM of the Dev edition, and can re-run setup OK. Jan Eliasen has also reported that things work OK on a 64-bit VMWare image running the same combination of software I am using.

Share This Post:

Posted On Sunday, September 26, 2010 1:58 PM | Feedback (4)

Thursday, September 23, 2010 #

BizTalk Server 2010 RTM

The BizTalk Server team announced today that BizTalk Server 2010 has RTMd. That's a relief, as my reputation was on the line regarding RTM before the end of the month. Just this Tuesday I confidently told a prospective customer to expect an imminent announcement.

I've been playing with BTS 2010 for a little while now. It has been my constant companion while writing several chapters for the forthcoming BizTalk Server 2010 Unleashed. It looks great. As well as bringing BTS up to date with Visual Studio 2010 and SQL Server 2008 R2, it introduces some welcome improvements. First and foremost, for developers, is a re-worked mapper. A decade ago, the mapper was the 'killer app' (as Clemens Vasters put it at the time) that sold BTS 2000 licences. A lot has changed since then, but the mapper, although much copied over the years, remains a vital part of BizTalk tooling. The new version is head and shoulders over the previous version and, frankly, over much of the competition as well. In the last few days, I've constructed a couple of medium-complexity maps in BTS 2009. I really, really wished I was using BTS 2010. Even though the maps were not by any means the biggest and ugliest I have worked on, things would still have been much easier with the new version. Better still, you can use the new mapper in conjunction with WCF/WF workflow services in .NET 4 (under the beta, you had to have AppFabric installed - I haven't tested this yet on the RTM version). It's a great indication of how, as time goes on, BizTalk Server is supporting and extending WCF/WF to an ever-greater degree. There is much more water to flow under this particular bridge.

One more point about the mapper. It seems the hilariously named 'Suggestive Match' feature in the beta (wink, wink) has been renamed 'Predictive Match'. They are a bunch of killjoys over there in Redmond, but I suppose double entrendres have no place in enterprise-level platforms.

The Trading Partner Management features have had a welcome makeover. Some of us older folk remember how BTS began life as the Commerce Interchange Pipeline (CIP) in Microsoft Site Server (ah, what a trip down memory lane!). BizTalk Server was originally conceived principally as a B2B product and has always had some TPM features. Over the years these have languished. BTS has excellent support for EDI standards (EDIFACT, X12 and AS2), but the TPM was relatively poor. That has changed in BTS 2010 with the introduction of a completely reworked model with new support for agreement templating and onboarding as well as new models for centralised management of identities, message types, validation, etc.

There are new features to please the operations guys who have to maintain BTS, including a new improved SCOM management pack, new rapid tuning features and data compression for backups. RFID Server has also been extended with a number of new event handlers and other features.

Looking through the official announcement, I am clearly going to have to revisit some of the adapter content I've recently been writing for BizTalk Server 2010 Unleashed. When looking at the beta, I discovered the SSL extensions to the existing FTP adapter, but MS keep on talking about a new FTPS adapter as if it is a separate item. This is currently confusing me. They also talk about new features of the File adapter which I didn't discover in the beta. [UPDATE 26/09/2010] The list of new features in the BizTalk Server Team blog is confusing. There is no new separate FTPS adapter, but simply a set of extensions to the existing FTP adapter, including SSL support. Also, I misinterpreted the list of features assuming that they referred to extensions to the File adapter. Not so. They are new features of the FTP adapter which I had actually discovered previously! The beta documentation clearly stated that the old SQL Server adapter has been removed from the product, but it was still there in the beta. I will be interested to see if it is in the RTM version. If it is, I understand that it will not be supported. [UPDATE 26/09/20010] The SQL Server adapter is still included in the product, BUT is no longer supported, except in certain cases. It appears that the help documentation has been tidied up in order to clarify the situation. Here is what it says:

"BizTalk Server no longer supports the SQL Adapter. We recommend you use Microsoft BizTalk Adapter for SQL Server to achieve the same functionality.

However, BizTalk Server supports the SQL Adapter with the components shipped with BizTalk Server, like EDI Control message SQL receive pipeline, or AS2 resend control message receive pipeline."

The old SOAP adapter, I believe, retains its deprecated status but is still supported. [UPDATE 26/09/20010] Yes, this is the case. This caused me some pain earlier this year when we discovered that a customer, new to BizTalk Server, needed to integrate with a central system via web services written in an older version of Apache Axis. Axis used to use RPC/Encoded as the default SOAP style, and I had to explain to the customer that, having spent all that money on a bright shiny new integration server, the only way it could support this part of the SOAP specification was through a deprecated adapter. The WCF adapters have no support for RPC. Also, publishing ASMX web services via BizTalk with RPC support is not handled well. You have to make changes to the code generated by the publishing wizard to get everything working as required. <rant>For BizTalk Server to fail to provide good support for a major, if unfashionable, part of the SOAP spec, and to only support RPC via a deprecated adapter verges on the incredible. In the rush to adopt pristine-pure WCF, there is always the fear that Microsoft could lose sight of the whole purpose of BizTalk Server and the value proposition it offers to companies. It is an integration server. It is there to cope with mess and confusion, mediocre and bad practice, delinquent systems and ancient protocols. Supporting things like RPC is what makes the cost of the license worthwhile.</rant>

I can't yet comment on the stability of the development tools in BTS 2010. I haven't experienced any major problems with the BTS 2010 beta, but I've yet to do any serious development in the new version. BTS 2009 dev tools were delivered in a poor, sometimes barely usable, state although subsequent hotfixes improved things significantly. The biggest issue in BTS 2009 remains the poor handling of references amongst multiple BizTalk projects. There were other problems as well. On my current project, I've been struggling just today with very peculiar and wrong behaviour when deploying a small cluster of BizTalk projects to a single application. I really hope that some headway has been made on these issues in BTS 2010 as they are an open sore in this otherwise excellent, stable and mature product.

So, welcome to BTS 2010, and congratulations to Microsoft on getting this version out of the door in the stated timeframe. Keep up the good work.

Share This Post:

Posted On Thursday, September 23, 2010 10:12 PM | Feedback (2)

RIP Quadrant. Adios, SSM Repository.

So it's official. Don Box has blogged an update on SQL Server Modelling. As widely rumoured, he has confirmed that Quadrant is dead. The shock is that the model repository has expired as well.

M is still hanging on in there, but we don't know anything more about what the future holds. For some time, the official word has been that it would arrive in conjunction with a future SQL Server release. I detect a possible change here, in that Don simply says 'We are continuing our investment in this technology and will share our plans for productization once they are concrete'. For my part, I maintain that tying 'M' to the SQL Server licence would be a really bad move. I know that every time Doug Purdy (who has recently left Microsoft to join Facebook) was challenged on this, he kept saying...no, SQL Server is a platform, not a product...watch this space. However, in the next breath it certainly sounded as if 'M' was in danger of being subsumed under a product license. No, no, no. Set it free to be used across Microsoft's platform. That platform is not SQL Server. It remains Windows, and for developers, it is Windows (on-premises and cloud) mediated through .NET.

Share This Post:

Posted On Thursday, September 23, 2010 6:24 PM | Feedback (0)

Wednesday, September 22, 2010 #

Optimising VirtualBox

I blogged a couple of weeks ago about my issues running VirtualBox. At the time, I believed I had found an issue involving SuperFetch. In reality, turning SuperFetch off, while helpful, did not address the underlying issue. I eventually worked out that my machine is overheating whenever CPU usage of 40-50% is sustained over several minutes. Several things, or combinations of things, can cause this on my machine, including running full virus-checking scans in Forefront, SuperFetch and, of course, running VirtualBox. Every time the temperature goes up to about 85^o, the power management throttles the CPU back to about 300MHz in order to let things cool down again. Once cooler (less than 80^o), the CPU switches back to full speed. A cycle is set up that repeats for as long as the relatively high CPU load continues. I use the RightMark CPU Clock Utility to monitor core temperature.

The fan works OK, and my machine is not clogged with dust and detritus. It appears that my Dell Vostro 1720 simply cannot cope with sustained load. The Vostro range has documented overheating problems in relation to the NVidia GPU. Dell ended up modifying the BIOS to keep the fan running constantly to try to keep things cool. Sure enough, my fan never seems to switch off, but just can't cope with sustained load. Short of convincing the company to buy me a new machine (some hope!), my only options are to adopt whatever strategy I can to reduce sustained CPU load to a minimum and to invest, at company expense, in a cooling pad.

How do I reduce sustained CPU load? Well, there are some challenges here. First, I avoid run full virus scans during working hours. Unfortunately, being a company machine, the policy is locked down and a full scan kicks off every Wednesday lunchtime. My only option has been to kill the scan and try to remember to run one manually before going to bed.

Turning SuperFetch off has helped. It doesn't really use a huge number of cycles, but it does increase CPU load for several minutes at a time at certain times of the day. In combination with other things, this can be enough to cause my machine to overheat. I haven't missed SuperFetch except when re-booting, which now takes a lot longer.

And now on to VirtualBox. I'm very dependent on running virtual images on my notebook for development, demonstrations, etc. Although Microsoft includes virtualisation with Windows 7, this is not an option for me. Microsoft only supports 32-bit images and, far worse, my experience was that most of my existing VPC 2007 images simply won't run under the new WinVPC. Hence, I, like everyone else I know, have now switched to Sun/Oracle VirtualBox which happily runs all my old 32 bit images and allows me to create 64-bit images. Indeed, Microsoft is quite explicit on their web site that they only see WinVPC as a mechanism for handling legacy 32-bit apps that won't run properly on Windows 7. I have yet to encounter any application that won't run on Windows 7, so this strategy does little for me.

Unfortunately, VirtualBox can be quite intensive in terms of CPU usage, and causes my machine to overheat. Even if the image is completely idle, VirtualBox can constantly consume 30-50% CPU. I've been struggling with this for some time. Here is a summary of strategies to reduce CPU load:

On a single processor, dual core machine, configure each image to use a single virtual processor. This seems to help. I haven't tested this on multi-processor boxes, so can't say if the rule should be 'always use a single virtual processor' or 'always use one less virtual processor than the physical processor/core count.

Avoid Oracle VirtualBox 3.2.8 and instead opt to use the old Sun VirtualBox 3.1.8 which is still available. The Sun version seems to be a bit less intensive than the current Oracle version. Oracle certainly introduced a number of changes and seems to burn through more CPU cycles.

And now, something I recently discovered. Try moving to SATA disk controllers. This has worked well for me. One of our system guys tells me that he suspects this better aligns VirtualBox with the controllers used on most modern notebooks, and cuts down the amount of work the machine has to do. I don't know if this is correct, but I certainly see a significantly reduced load on the machine.

To move to SATA disk controllers, you need to stop you image and then go to 'Settings/Storage'. Under the 'Storage Tree' panel, click the 'Add Controller' button and choose 'Add SATA controller'. Now boot the image and make sure it has a valid network connection with Internet access. To load the required drivers, go to the Intel web site and download the latest version of their Matrix Storage Manager. You can currently find this on this page:

http://downloadcenter.intel.com/SearchResult.aspx?lang=eng&ProductFamily=Chipsets&ProductLine=Chipset+Software&ProductProduct=Intel%c2%ae+Rapid+Storage+Technology+(Intel%c2%ae+RST)

Once the manager is installed, shut down the image. Then go back to 'Settings/Storage', remove your hard disk from its current controller and add it to the SATA controller. Reboot you image, and hopefully things will improve.

Finally, if you have the same problems I have, invest in a cooling pad. Mine arrive at lunchtime today just in time for me to test it during a full Forefront scan. I placed the notebook on the pad, plugged it into my USB port (to run the fan) and watched the temperature reduce down by about 12/13^o. For the next few hours it ran the full scan without going much above 60^o, which is well inside the safety zone. At the same time, I was running, and using, VirtualBox. The scan is complete and I am down to about 54^o. This still strikes me as a tad on the warm side, and I have to say I think there is probably some underlying fault on my machine, but our systems guys feel differently. So, for the time being, armed with VirtalBox optimisations and a cooling pad, I can once more enjoy a productive relationship with my notebook.

Share This Post:

Posted On Wednesday, September 22, 2010 6:49 PM | Feedback (2)

Charles Young

News

Twitter

Article Categories

Archives

Post Categories

Image Galleries

Alternative Feeds

BizTalk Bloggers

BizTalk Sites

CEP Bloggers

CMS Bloggers

Fun

Other Bloggers

Rules Bloggers

SharePoint Bloggers

Utilities

WF Bloggers