Code Gallery

Passthrough Objects – Duck Typing++

EltonStoneman — Thu, 13 May 2010 18:51:43 GMT

[Source: http://geekswithblogs.net/EltonStoneman]

Can't see a genuine use for this, but I got the idea in my head and wanted to work it through. It's an extension to the idea of duck typing, for scenarios where types have similar behaviour, but implemented in differently-named members.

So you may have a set of objects you want to treat as an interface, which don't implement the interface explicitly, and don't have the same member names so they can't be duck-typed into implicitly implementing the interface. In a fictitious example, I want to call Get on whichever ICache implementation is current, and have the call passed through to the relevant method – whether it's called Read, Retrieve or whatever:

A sample implementation is up on github here: PassthroughSample.

This uses Castle's DynamicProxy behind the scenes in the same way as my duck typing sample, but allows you to configure the passthrough to specify how the inner (implementation) and outer (interface) members are mapped:

var setup = new Passthrough();

var cache = setup.Create("PassthroughSample.Tests.Stubs.AspNetCache, PassthroughSample.Tests")

.WithPassthrough("Name", "CacheName")

.WithPassthrough("Get", "Retrieve")

.WithPassthrough("Set", "Insert")

.As<ICache>();

- or using some ugly Lambdas to avoid the strings :

Expression<Func<ICache, string, object>> get = (o, s) => o.Get(s);

Expression<Func<Memcached, string, object>> read = (i, s) => i.Read(s);

Expression<Action<ICache, string, object>> set = (o, s, obj) => o.Set(s, obj);

Expression<Action<Memcached, string, object>> insert = (i, s, obj) => i.Put(s, obj);

ICache cache = new Passthrough<ICache, Memcached>()

.Create()

.WithPassthrough(o => o.Name, i => i.InstanceName)

.WithPassthrough(get, read)

.WithPassthrough(set, insert)

.As();

- or even in config:

ICache cache = Passthrough.GetConfigured<ICache>();

...

<types>

<typename="PassthroughSample.Tests.Stubs.ICache, PassthroughSample.Tests"

passesThroughTo="PassthroughSample.Tests.Stubs.AppFabricCache, PassthroughSample.Tests">

<membername="Name"passesThroughTo="RegionName"/>

<membername="Get"passesThroughTo="Out"/>

<membername="Set"passesThroughTo="In"/>

</members>

</type>

Possibly useful for injecting stubs for dependencies in tests, when your application code isn't using an IoC container. Possibly it also has an alternative implementation using .NET 4.0 dynamic objects, rather than the dynamic proxy.

Code Gallery Reshuffle

EltonStoneman — Thu, 26 Nov 2009 02:16:37 GMT

[Source: http://geekswithblogs.net/EltonStoneman]

I've been posting code samples and walkthroughs on MSDN Code Gallery for a while now – keeping them distinct from CodePlex which I use for fully-fledged projects and tools. CodePlex uses TFS for source control, but the Code Gallery only lets you post up individual files, so until now I've been creating releases as ZIP files containing the source.

Now I've centralised the source for all those samples in a public repository on github at sixeyed/codegallery. The clone URL is: git://github.com/sixeyed/codegallery.git. If you're new to git, the Illustrated Guide to Git on Windows is a good starting point.

For future work, I'll continue posting the source ZIPs as resources on MSDN Code Gallery, but the master will be on github.

Processing Excel uploads with BizTalk and nServiceBus

EltonStoneman — Sat, 21 Nov 2009 01:57:56 GMT

[Source: http://geekswithblogs.net/EltonStoneman]

Following on from the sample for processing Excel uploads with nServiceBus, I have a comparable version using BizTalk on MSDN Code Gallery here: BizTalk and nServiceBus Excel Upload. The BizTalk (2006 R2) sample uses a FILE receive port with a simple pipeline component to disassemble the Excel file into separate messages. A SQL send port subscribes to the messages, and with an outbound map calls the AdventureWorks stored procedure to insert each product.

Processing Excel in BizTalk is nothing new, but I wanted to do a comparison against the nServiceBus example, and also see how BizTalk and nServiceBus could be integrated. My original thinking was that the trigger in the nServiceBus solution relies on a FileSystemWatcher, which is less reliable and less flexible than BizTalk’s FILE adapter. A hybrid solution could use the FILE adapter to receive and parse the upload, then send each row as an AddProduct message to MSMQ, which the nServiceBus handler subscribes to.

Running this on the same environment (a Windows Server 2003 VM running under VirtualBox in Ubuntu 9.04), the BizTalk solution processes the 3,500 row Excel file in 3 mins 10 seconds (compared to 4m 15s for the distributed NSB running with 5 threads), and the 12,000 row file in 11m 14s (compared to 14m 0s). I was surprised to find the BizTalk solution running more quickly, as the original NSB was using non-recoverable messaging, so all messages were in memory, while BizTalk had the latency of writing to the message box.

Performance Comparison

This is not an intended to be a thorough benchmark of NSB and BizTalk – both sample projects are basic, untuned implementations, and the tests are on a single box rather than two or three. But for comparison, I ran the 3,500 row upload repeatedly under different configurations:

1. NSB with distributor – host and distributor using 20 threads

2. NSB with distributor – host and distributor using 20 threads, recoverable messaging

3. NSB without distributor – host using 20 threads

4. NSB without distributor – host using 20 threads, recoverable messaging

5. BizTalk FILE receive and SQL send

6. NSB parsing Excel file, BizTalk subscribing to AddProduct messages – MSMQ receive using batches of 100

7. BizTalk parsing Excel file, NSB without distributor subscribing to AddProduct messages – host using 20 threads

Which gave these averaged results:

Boosting the number of threads dramatically improved the NSB performance, and removing the distributor halved the duration. I would expect that running the distributor on a separate node would yield similarly good results on each processing node. The hybrid BizTalk/NSB configurations were the slowest – to be expected, as you have the latency of the message box and the latency of MSMQ saving to disk. NSB configurations without recoverable messaging were only marginally slower than the recoverable version, which seems incorrect based on Udi Dahan’s performance benchmark, so I’ll need to look into that further.

There’s plenty of scope for improving performance in both solutions. BizTalk can be endlessly tuned (the 2004 guidelines still apply as a good starting point). By modifying the NSB solution to send the AddProduct messages in one opration and using 80 threads for the handler, duration fell to 25 seconds – 140 messages per second.

Integrating BizTalk and nServiceBus

Despite having the worst performance, integrating BizTalk and nServiceBus is a viable option which may be very useful in some cases. It would allow you to leverage BizTalk’s adapter suite and mapping functionality to join LOB systems into an nServiceBus estate. Integration is actually very straightforward. By default NSB uses MSMQ, so to publish messages from BizTalk to an NSB subscriber just means configuring an MSMQ send port with the expected queue, and mapping your message to NSB format – which is an envelope containing one or messages:

<?xml version="1.0" ?>

<BatchSourcePath>E:\ExcelUpload\1.0.0.0\Binaries\Drops\ProductUpload.xls</BatchSourcePath>

<OriginatorDestination>ExcelUpload.Client.InputQueue@WIN2003R2-VM</OriginatorDestination>

<Name>new product 58</Name>

</AddProduct>

</Messages>

The MSMQ adapter lets you specify Recoverable and Transactional flags, so your BizTalk-generated messages have the same durability options. When an NSB handler is listening at the queue, it processes the BizTalk messages in the same way as NSB messages.

Publishing NSB messages to BizTalk is trickier, as the Send and Publish methods from NSB check to see if there are any subscribers before they write to the queue (compare this to the Notify method in RhinoServiceBus). If a BizTalk MSMQ receive port is the only handler, no subscribers will be registered with NSB and the message won’t be published. So you need to either call Send with a named queue (Bus.Send<AddProduct>("ExcelUpload.AddProductService.1.InputQueue", m =>...), or send a subscription message from BizTalk to register a subscriber. In the the second option you’d need to specify the message type in the subscription:

<?xml version="1.0" ?>

<string>ExcelUpload.Messages.StartBatchUpload, ExcelUpload.Messages, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null</string>

Running the Sample

(If you haven’t seen the original NSB sample, have a read as it details the pre-requisites).

To run the BizTalk solution, download ExcelUpload.BizTalk.Binaries.zip, unzip it and:

· Copy ExcelUpload.PipelineComponents.dll to your pipeline components directory – e.g. C:\Program Files\Microsoft BizTalk Server 2006\Pipeline Components

· If you haven’t done so from the previous sample, run uspInsertProduct.CREATE.sql

· Run uspInsertProduct.CREATESYNONYM.sql

· Import the MSI into BizTalk

· Run the MSI

· Start send port SendInsertProduct.SQL (you’ll need to change the connection details)

· Enable receive location ReceiveStartUpload.FILE.XLS (monitors c:\drops\ExcelUpload)

· Drop an Excel file

As before, you’ll need to clear down the database between runs if you drop the same file repeatedly.

If you want to run the hybrid solutions, you should use the binaries in the new download rather than the original, as these run without a distributor. Run start.cmd and there are three windows – Client (the file watcher), Host (the StartBatchUpload handler) and AddProductService (this is the AddProduct handler).

To run configuration 6 – NSB parsing the file and BizTalk handling the AddProduct messages:

· Run start.cmd

· Kill the AddProductService console

· Disable receive location ReceiveStartUpload.FILE.XLS

· Enable receive location ReceiveAddProduct.MSMQ

· Unenlist send port SendAddProduct.MSMQ

· Start send port SendInsertProduct.SQL

· Drop an Excel file

To run configuration 7 – BizTalk parsing the file and NSB handling the AddProduct messages:

· Run start.cmd

· Kill the Host and Client consoles

· Enable receive location ReceiveStartUpload.FILE.XLS

· Disable receive location ReceiveAddProduct.MSMQ

· Start send port SendAddProduct.MSMQ

· Unenlist send port SendInsertProduct.SQL

· Drop an Excel file

Source Code

The source and referenced assemblies are in ExcelUpload.BizTalk.Source.zip. There is a VS 2008 solution for the nServiceBus components, and a VS 2005 solution for the BizTalk components.

Processing an Excel upload with nServiceBus

EltonStoneman — Wed, 18 Nov 2009 09:46:13 GMT

[Source: http://geekswithblogs.net/EltonStoneman]

We’ve had a couple of projects recently with similar requirements to process an Excel file as a batch upload of data. One was a BizTalk project where the FarPoint Spread pipeline component was a good fit; the other was a Web app where we put together a custom parser based on the open-source ExcelDataReader. The custom solution was appropriate for the expected size of upload files, but wouldn’t scale well to deal with large files quickly, so I wanted to look at a distributed alternative using nServiceBus. My sample implementation is on MSDN Code Gallery here: nServiceBus Excel Upload. I’ll look at a comparative BizTalk solution in a future post.

If you haven’t come across nServiceBus, it’s a queue-based messaging framework which is inherently asynchronous. “Scalability and reliability are in its DNA”, and it has some impressive case studies. Using nServiceBus you can set up a simple publish-subscribe architecture between nodes, or a load-balanced architecture with a central distributor. In the distributed version, the upload sample looks like this:

(Note that the diagram represents the bus as a separate entity, in reality it’s distributed among the queues of all the nodes. The diagram also omits the distributor).

In nServiceBus, services are requested by publishing messages onto the bus. Requests are fulfilled by a handler which subscribes to a type of message. The Excel upload sample takes a workbook which contains a set of products and uploads them to the AdventureWorks database. There are three types of message:

StartBatchUpload – published when a file has been received and is ready to be processed; subscriber does some basic validation on the Excel data structure, and then for each row in the worksheet publishes an AddProduct message;
AddProduct – subscriber maps the product defined in the message to a stored procedure call which inserts the new product. When the last product in the batch is reached, sends a BatchStatusChanged message to the original publisher of the StartBatchUpload message;
BatchStatusChanged – logs the status change and renames the Excel upload file.

This is a basic example, more validation would be expected, but the workflow is representative. Parsing the Excel file is done quickly, allowing for any number of nodes to participate in the resource-intensive work of creating the products. Using a single host with 5 threads, an Excel file with 3,500 rows takes just over 4 minutes to process on a dev laptop. That’s 13 messages per second which is nothing special, but this is on a single host which is also running the distributor and SQL Server. The processing host has a flat memory profile (consistently around 40Mb) and runs at less than 20% CPU. The distributor takes around 15% CPU, and MSMQ another 15%.

For a much larger upload – 12,000 rows – the processing and memory profile is the same, and the upload takes around 14 minutes (~14 messages per second) on the same infrastructure.

Running the Sample

Access to a SQL Server instance with the AdventureWorks sample database installed is a pre-requisite. You’ll need to add the new stored procedure with uspInsertProduct.CREATE.sql. The connection string used by the host is specified in ExcelUpload.Host.exe.config (defaults to unnamed local instance).

You’ll need MSMQ running on all nodes. Queues are specified in configuration and are created by nServiceBus if they don’t exist – an exception is the storage queue for the distributor which needs to be manually created, this PowerShell snippet will do it:

[Reflection.Assembly]::LoadWithPartialName("System.Messaging")

[System.Messaging.MessageQueue]::Create(".\Private$\distributorStorage", $true)

Unzip the file ExcelUpload.Binaries.zip. You’ll have a batch file – start.cmd – and five subdirectories – Client, Host, Distributor, SampleFiles and Drops. Run start.cmd, check the console screens for errors, then copy one of the Excel files from SampleFiles to Drops. You should see activity in the host, client and distributor console screens, and new rows being added to the [Production].[Product] table.

If you drop the same file twice, the unique key on Products will be violated, so the upload will error. On a fresh install there are under 1000 products, so this resets the table to the default state:

delete [Production].[Product] where ProductID > 999;

Implementation Details

The sample uses the release version of nServiceBus – 1.9 – as the distributor was broken in the 2.0 beta at the time of writing.

The two console apps run the “client” (which monitors a configured file location for an Excel drop, and publishes the StartBatchUpload message), and the “host” (which subscribes to StartBatchUpload and publishes AddProduct and BatchStatusChanged messages). Both use Topshelf so they can run as a console, or can be installed as a Windows service (e.g. ExcelUpload.Client.exe /install).

If you want to run several hosts on the same machine, they will need to use different queues. Copy the whole of the Host directory, and modify ExcelUpload.Host.exe.config to specify a unique queue name:

InputQueue="ExcelUpload.Service.1.InputQueue"

Then run ExcelUpload.Host.exe from all the copied locations, and you’ll see the console hosts sharing the message processing when a file is dropped.

Duck Typing with Castle

EltonStoneman — Wed, 21 Oct 2009 01:59:24 GMT

[Source: http://geekswithblogs.net/EltonStoneman]

David Meyer has a neat duck typing library for .NET which I’ve been looking at as I had a requirement to treat objects of various types in the same way. They all expose a “Value” property, but don’t implement an interface.

The library does more than I needed so I put together a quick duck typing implementation which just lets me treat an instance of an object which tacitly implements an interface, as a true implementation of the interface. In this example I can access BespokeDateTime and BespokeInt as IHasValue:

Using Castle’s Dynamic Proxy, this is simple to do – just requiring you to generate an interface proxy with a custom interceptor. The interceptor invokes the interface method on the real object, so you effectively wrap an instance in an interface it doesn’t explicitly implement:

BespokeDateTime bespoke = new BespokeDateTime(new DateTime(2000, 1, 1));

IHasValue duck = DuckType.As<IHasValue>(bespoke);

Console.WriteLine(duck.Value);

- or using an object extension to hide the implementation:

bespoke.As<IHasValue>().Value;

Iterating over collections is much cleaner with duck typing – if I have a collection that contains both BespokeDateTime and BespokeInt objects, to get all their values I can use:

foreach (object obj in GetObjects())

{

Assert.IsNotNull(DuckType.As<IHasValue>(obj).Value);

}

- compared to casting each object:

foreach (object obj in GetObjects())

{

BespokeDateTime bespokeDateTime = obj as BespokeDateTime;

if (bespokeDateTime != null)

{

Assert.IsNotNull(bespokeDateTime.Value);

}

else

{

BespokeInt bespokeInt = (BespokeInt) obj;

Assert.IsNotNull(bespokeInt.Value);

}

Use with caution over large collections, as there is a noticeable performance hit in using the duck type. Over 500,000 items the duck-typed code takes 40-50 times as long as the casting code, averaging 8 seconds to 0.2 seconds, with duck-typing by extension taking marginally longer again:

The full code (all 34 lines – including validation that the type to duck does tacitly implement the interface) is on MSDN Code Gallery here: Duck Typing with Castle.

WCF Personalization Provider for ASP.NET

EltonStoneman — Tue, 15 Sep 2009 13:49:24 GMT

[Source: http://geekswithblogs.net/EltonStoneman]

The ASP.NET Web Part framework uses a plug-in framework for loading and saving personalization data to a persistent source. Out of the box, a SQL Server data source is provided (via the aspnet_regsql tool), along with a SQL personalization provider – specified in the system.web configuration section:

<add name="SqlPersonalizationProvider"

type="System.Web.UI.WebControls.WebParts.SqlPersonalizationProvider"

connectionStringName="local"

applicationName="WcfPersonalizationSample" />

It's painless to set up and gets you running Web Part-enabled sites quickly. But if your Web servers and database servers are separated by a physical boundary with app servers in between, you can't use the SQL provider directly from the ASP.NET application and will need to write a custom personalization provider.

Exposing the existing SQL personalization store through a custom WCF provider is a straightforward option – so your app server hosts a WCF service which itself uses the SQL personalization provider, and the Web server uses the WCF service through the custom provider. Although the implementation is fairly trivial, there are a couple of issues to work around:

PersonalizationProvider is an abstract class which requires you to implement methods with out and ref parameters. You'll need to write methods with different signatures to work over WCF;
SqlPersonalizationProvider has methods for loading personalization data as byte arrays, which are useful for the WCF service. To access them you'll need another custom provider which inherits from SqlPersonalizationProvider, as the underlying methods are protected.

I've worked this through as a sample on MSDN Code Gallery: WCF Personalization Provider Sample.

To use it, the WCF service host needs to be configured to use the custom provider which wraps the SQL store, and a connection string entry for the data store:

<add name="local"

connectionString="Data Source=.;Initial Catalog=WcfPersonalizationSample;Integrated Security=True"

providerName="System.Data.SqlClient" />

</connectionStrings>

<system.web>

<add name="PersonalizationServiceProvider"

type="WcfPersonalizationProvider.PersonalizationServiceProvider"

connectionStringName="local"

applicationName="WcfPersonalizationSample" />

</providers>

</personalization>

</webParts>

...

(Note that this is using ASP.NET functionality in the WCF service, but you do not need to set the WCF service up for ASP.NET compatibility mode.)

The Web site need to be configured to use the custom provider which accesses the WCF service, and a service client entry for the WCF service:

<add name="WcfPersonalizationProvider"

type="WcfPersonalizationSample.ServiceClient.WcfPersonalizationProvider"

applicationName="WcfPersonalizationSample" />

</providers>

</personalization>

</webParts>

...

<endpoint address="http://localhost/PersonalizationService.svc"

binding="basicHttpBinding"

contract="PersonalizationService.IPersonalizationService"

name="BasicHttpBinding_IPersonalizationService" />

</client>

Both providers need to specify the same applicationName value, and your Web Part personalization, for shared and user scopes, will work without the Web servers accessing the database.

FluentHtmlTextWriter

EltonStoneman — Thu, 10 Sep 2009 18:32:28 GMT

[Source: http://geekswithblogs.net/EltonStoneman]

Keeping up the fluent work, I've put together a fluent interface which wraps the framework HtmlTextWriter. For ASP.NET MVC, this makes generating HTML in extension methods to HtmlHelper safer than string.Format() and more readable than HtmlTextWriter:

public static string Image(this HtmlHelper helper, string imageRelativeUrl, string altText)

{

return FluentHtmlTextWriter.Begin()

.WriteTag(HtmlTextWriterTag.Img)

.WithAttribute(HtmlTextWriterAttribute.Src, GetImageUrl(imageRelativeUrl))

.WithAttribute(HtmlTextWriterAttribute.Alt, altText)

.End();

}

The equivalent using HtmlTextWriter is:

public static string Image(this HtmlHelper helper, string imageRelativeUrl, string altText)

{

StringBuilder htmlBuilder = new StringBuilder();

HtmlTextWriter writer = new HtmlTextWriter(new StringWriter(htmlBuilder));

writer.AddAttribute(HtmlTextWriterAttribute.Src, GetImageUrl(imageRelativeUrl));

writer.AddAttribute(HtmlTextWriterAttribute.Alt, altText);

writer.RenderBeginTag(HtmlTextWriterTag.Img);

writer.RenderEndTag();

writer.Flush();

return htmlBuilder.ToString();

}

Unlike the framework writer, you're not constrained to specify attributes in any particular order, and tags which aren't nested can be written with a single WriteTag(), rather than BeginTag() and EndTag() calls. The writer copes with multi-level tag hierarchies - this sample builds a Superfish CSS menu:

FluentHtmlTextWriter writer = FluentHtmlTextWriter.Begin();

writer.BeginTag(HtmlTextWriterTag.Ul)

.WithAttribute(HtmlTextWriterAttribute.Id, "menu.Name")

.WithAttribute(HtmlTextWriterAttribute.Class, "sf-menu sf-vertical");

.BeginTag(HtmlTextWriterTag.Li)

.BeginTag(HtmlTextWriterTag.A)

.WithAttribute(HtmlTextWriterAttribute.Class, "sf-with-ul")

.WithAttribute(HtmlTextWriterAttribute.Href, "#")

.WithValue("Link 1")

.WriteTag(HtmlTextWriterTag.Span)

.WithAttribute(HtmlTextWriterAttribute.Class, "sf-sub-indicator")

.WithValue("»")

.EndTag()

.EndTag();

string html = writer.End();

Assert.AreEqual("<ul id=\"menu.Name\" class=\"sf-menu sf-vertical\">\r\n\t<li><a class=\"sf-with-ul\" href=\"#\">Link 1<span class=\"sf-sub-indicator\">»</span></a></li>\r\n</ul>",

html);

The HtmlTextWriter equivalent is unthinkable.

Patrik Hägne has an alternative fluent HtmlTextWriter implementation, which is nicely put together, but I wanted slightly different functionality. Firstly I wanted to get the HTML string from the writer directly, without needing to instantiate a StringBuilder and StringWriter. Secondly I wanted minimal new code – Patrik uses a separate class to manage writing attributes, and has specific functions for known tag types. Thirdly, I didn't really like Patrik's syntax, with the need to specify the tag type when you write an end tag – and when tags aren't nested, I wanted to write them in a single unit:

string html = FluentHtmlTextWriter.Begin()

.WriteTag(HtmlTextWriterTag.Span)

.WithAttribute(HtmlTextWriterAttribute.Id, "id_span")

.WithValue("contents_span")

.End();

Assert.AreEqual("<span id=\"id_span\">contents_span</span>", html);

My version is on MSDN Code Gallery here: FluentHtmlTextWriter. It works by building up a list of actions when you start writing a tag, and flushing them in the correct order to an internal HtmlTextWriter when you start writing the next tag. When you call End() it flushes the internal writer and outputs its contents. An alternative constructor lets you write directly to an output stream, in which case you can Flush() the writer and don't need to call End().

The current implementation only deals with basic tag, attribute and style functionality, and doesn't include optional HTML encoding overloads. It's only 75 lines of code, and extending it should be trivial.

There is a negligible performance hit in using FluentHtmlTextWriter – running the image tag generation code above 20,000 times on my dev machine, the fluent version takes between 0.01and 0.02 seconds longer than the framework version (on average 0.07 seconds compared to 0.05).

Fluent DAL Mapping

EltonStoneman — Tue, 08 Sep 2009 01:32:20 GMT

If you spend some time using Fluent NHibernate, you’ll want to use its neat style of mapping for all data access, even when you’re working against traditional DALs. I’ve put a sample up on MSDN Code Gallery for this scenario, using a fluent style of mapping between domain objects and data readers populated by stored procedure calls.

The interface is very similar to FNH, with a mapping class used for each domain entity – this is a simple mapping class:

public class PostCodeMap : DataReaderMap<PostCode>

{

public PostCodeMap()

{

Map(x => x.InwardCode, "PS_IN");

Map(x => x.OutwardCode, "PS_OUT");

}

The string constants define the column names expected in the data reader, and the base class takes care of running the map. For more complex entities, DataReaderMap also includes mapping via a conversion function:

Map<string, bool>(x => x.Activated, "AccountActivated", Legacy.FromBoolean);

- where the mapping specifies to and from types, and a delegate to invoke for the conversion (in this case, a method which converts a legacy string representation of a boolean – Y/N – to a bool).

For composite objects, the base map includes referencing, for cases where the child entity is loaded from the same data reader as the parent entity:

References(x => x.Address, new AddressMap());

Where the child entity is read from a separate data reader, then the composition needs to be done outside of the map in a repository or an assembler component. The Load class isolates calling the stored procedure, so you get a fluent interface for populating domain objects, and collections of objects:

public User GetUser(Guid userId)

{

//populate basic details:

User user = Fluently.Load<User>().With<UserMap>()

.From<GetUser>(i => i.UserId = userId,

x => x.Execute());

//add accounts:

user.Accounts = Fluently.Load<List<Account>>().With<AccountMap>()

.From<GetUserAccounts>(i => i.UserId = userId,

x => x.Execute());

return user;

}

The From() call specifies the type of stored procedure, an action to populate the object, in this case setting up the UserId parameter, and a function to call which returns a data reader, in this case Execute().

The full sample is on the gallery here: Fluent DAL Mapping . It’s a simple task to extend the sample to load datasets, or to populate update procedure calls from entities. If your database is sufficiently conventional this could be extended to provide FNH-style auto-mapping, reflecting over entities and using conventions to map properties to column names.

Using PowerShell in BizTalk Post-Processing Scripts

EltonStoneman — Mon, 08 Jun 2009 14:58:18 GMT

[Source: http://geekswithblogs.net/EltonStoneman]

Often in BizTalk deployments you need to do additional work after installation. Typically your full install process may need to:

Install BizTalk artifact assemblies to the GAC
Install application dependencies to the GAC
Register an application source name in the registry, for logging to the Event Log
Create FILE send or receive locations on the local filesystem
Add application store configuration settings to Enterprise Single Sign-On (SSO)
Add log4net configuration settings to BTSNTSvc.exe.config

You can achieve this with a single BizTalk installer by configuring resources and post-processing scripts, and exporting an MSI from the application. Various scripting languages are supported in BizTalk installations (batch files, VBScript etc.), except the most logical – PowerShell, which gives first-class support for the filesystem, the registry, XML files and .NET objects. You can still use PowerShell by including scripts as resources, and using a batch file as the post-processing script, which acts as a harness to call the PowerShell scripts.

This walkthrough addresses all the points above. The completed BizTalk application is on MSDN Code Gallery here: BizTalk PowerShell Deployment Sample – import and install the MSI to deploy with the PowerShell script, or browse the ZIP file to see the scripts and resources.

1. Install BizTalk artifact assemblies to the GAC

This is straightforward, set the resource option "Add to the global assembly cache on MSI file install" to true – this happens by default if you add a BizTalk Assembly resource in the Administration Console:

Using the command line though, this is not the default option so you need to explicitly set –Options:GacOnInstall in BTSTask:

btstask AddResource

-ApplicationName:PowerShellSample

-Type:BizTalkAssembly

-Options:GacOnInstall

-Source:PowerShellSample.Schemas.dll

-Destination:%BTAD_InstallDir%\PowerShellSample.Schemas.dll

2. Install application dependencies to the GAC

As 1), except the resource type is System.BizTalk:Assembly (in BTSTask you can omit "System.BizTalk"). The command requires the same flag to add to the GAC on install:

btstask AddResource

-ApplicationName:PowerShellSample

-Type:Assembly

-Options:GacOnInstall

-Overwrite

-Source:.\Dependencies\SSOConfig.dll

-Destination:%BTAD_InstallDir%\Dependencies\SSOConfig.dll

In this case, I'm installing the SSOConfig assembly (from SSO Config Tool) which provides static .NET classes for accessing the SSO application configuration store. The Overwrite flag is set in case the resource already exists in another application.

3. Register an application source name in the registry, for logging to the Event Log

To log to the Application event log with your own source name, you need to add a registry key with the app name, and the name of the handler:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Eventlog\Application\PowerShell.Sample

In PowerShell, this is done in using the New-Item cmdlet to create the key, and New-ItemProperty to set the key value:

New-Item -Path 'HKLM:\SYSTEM\CurrentControlSet\Services\Eventlog\Application\PowerShellSample' -Force

New-ItemProperty -Path 'HKLM:\SYSTEM\CurrentControlSet\Services\Eventlog\Application\PowerShellSample' -Name 'EventMessageFile' -PropertyType ExpandString -Value 'C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\EventLogMessages.dll' –Force

(HKLM: is a PowerShell drive mapped to HKEY_LOCAL_MACHINE, and the –Force flag overwrites existing values).

To execute the PowerShell script on install, we need a batch file which BizTalk can run as a post-processing script. The batch file is very simple, just separating install and uninstall logic to individual PowerShell scripts, and redirecting script output to a log file:

cd "%BTAD_InstallDir%\Deployment"

if "%BTAD_InstallMode%" == "Install" ( powershell ".\PowerShellSample.Install.ps1" >> PowerShellSample.Install.ps1.log )

if "%BTAD_InstallMode%" == "Uninstall" ( powershell ".\ PowerShellSample.Uninstall.ps1" >> PowerShellSample.Uninstall.ps1.log )

Both the CMD and PS1 files need to be added as resources to the BizTalk application. The PS1 files are of type BizTalk:File, and the CMD harness is of type BizTalk:PostProcessingScript:

btstask AddResource

-ApplicationName:PowerShellSample

-Type:File

-Source:.\Deployment\PowerShellSample.Install.ps1

-Destination:%BTAD_InstallDir%\Deployment\PowerShellSample.Install.ps1

btstask AddResource

-ApplicationName:PowerShellSample

-Type:PostProcessingScript

-Source:.\Deployment\PowerShellSample.PostProcessing.cmd

-Destination:%BTAD_InstallDir%\Deployment\PowerShellSample.PostProcessing.cmd

4. Create FILE send or receive locations on the local filesystem

If you need to create static file locations, the same New-Item cmdlet is used with the filesystem provider. Specify the full path for the directory and any intermediate directories will be created if they don't exist. Use the –Force flag to suppress warnings if the directory already exists:

New-Item –Path 'c:\receiveLocations\x\y\z' -ItemType Directory –Force

Note that the resources in the BizTalk application are copies rather than references, so if you modify your PS1 files, you'll need to update the resource (in the Administration Console, select the resource and use Modify... Refresh, or re-run the BTSTask command).

5. Add settings to Enterprise Single Sign-On (SSO)

If you're using SSO to store group-wide application config, you can create or export an XML file of the settings using SSO Config Tool. We add the .ssoconfig file as a File resource to the application, then in the install script use PowerShell to call a .NET method to import the settings using the SSOConfig.SSOApplication class. The SSOConfig assembly is a resource which has already been deployed to the GAC by this point in the installation:

[Reflection.Assembly]::Load('SSOConfig, Version=1.1.0.0, Culture=neutral, PublicKeyToken=656a499478affdaf')

$configPath = [IO.Path]::Combine($env:BTAD_InstallDir, 'Deployment\PowerShellSample.ssoconfig')

$app = [SSOConfig.SSOApplication]::LoadFromXml($configPath)

$app.SaveToSSO()

Note that the PowerShell script has access to all the environment variables set by BizTalk on the install – accessed by prefixing $env: to the variable name, as we do here to get the installation directory from the installer ($env:BTAD_InstallDir).

6. Add settings to BTSNTSvc.exe.config

Modifying XML is straightforward in PowerShell. We want to configure an Event Log appender in the BizTalk service config file by adding the following XML:

</configSections>

</layout>

</appender>

<appender-ref ref="Sixeyed.CacheAdapter.EventLogAppender" />

</logger>

</log4net>

The Get-ItemProperty cmdlet can read the BizTalk install path from the registry, then Get-Content reads the file – casting it to XML for subsequent processing:

$installPath = Get-ItemProperty -Path 'HKLM:\SOFTWARE\Microsoft\BizTalk Server\3.0' -Name 'InstallPath'

$btsConfigPath = [IO.Path]::Combine($installPath.InstallPath, 'BTSNTSvc.exe.config')

$xml = [xml] (Get-Content $btsConfigPath)

On a fresh install, the config file is quite bare and doesn't include a <configSections> element, so in that case we need to add both <configSections> and <log4net> nodes. We can't guarantee that other solutions haven't already modified the config file though, so <configSections> may exist, and <log4net> may also exist – in which case, we just need to add our specific appender and logger values (log4net allows you to define multiples of these in config, and we specify names which we can expect to be unique).

To achieve this, the script checks for each element first, creates it if it doesn't exist, then adds the specific settings:

$configSections = $xml.SelectSingleNode('configuration/configSections')

if ($configSections -eq $null)

{

$configSections = $xml.CreateElement('configSections')

$firstChild = $xml.configuration.get_FirstChild()

$xml.configuration.InsertBefore($configSections, $firstChild)

}

$log4netSection = $configSections.SelectSingleNode('section[@name="log4net"]')

if ($log4netSection -eq $null)

{

$log4netSection = $xml.CreateElement('section')

$log4netSection.SetAttribute('name', 'log4net')

$log4netSection.SetAttribute('type', 'log4net.Config.Log4NetConfigurationSectionHandler, log4net, Version=1.2.10.0, Culture=neutral, PublicKeyToken=1b44e1d426115821')

$configSections.AppendChild($log4netSection )

}

...

Finally the updates are saved over the original file:

$xml.Save($btsConfigPath)

Limitations

The main limitation with any post-processing script, is that the target environment selected for the install is not available. If you have multiple bindings files, the environment selected at runtime is only alive for the duration of the MSI import – the install has no reference to it, and there's no record made in the management database (not that I can see, please correct me if there is). This means you can't switch your script based on environment (e.g. to use different SSO config settings for System Test and Production). If that's a serious restriction you may prefer to create different MSIs per-environment in your build process, each containing the correct bindings file and scripts.

Specific to this approach, you need to have PowerShell installed on all the target machines, and configured to allow script execution (by default, scripts are not permitted to execute, as a security measure). Hopefully this is becoming the norm. Security also needs to be considered – the sample app writes to the registry and to SSO, so the installing context needs to have explicit permissions. The BizTalk installer runs under a separate security context from the installing user (by a trial-and-error process, this is NT AUTHORITY\ANONOYMOUS LOGON in my Server 2003 VM), so if you're amending SSO you'll need to set your SSO Administrators group membership correctly.

Benefits

The completed PowerShell scripts should be straightforward to read and maintain. All the post-installation requirements are implemented using a single technology, and many of the functions are reusable and could easily be parameterised and moved to a central script. The script is easy to test outside of the installer runtime, either manually using a batch file as a test harness (which sets up the relevant environment variables and then calls the post-processing file), or worked into an automated unit test.

The approach is not limited to BizTalk installations, so similar tasks for .NET deployments which are currently done with custom assemblies or Wix script can be isolated in the same way. With BizTalk and .NET installs using the same technology, you'll build up a library of high-quality, reusable PowerShell scripts.

I also like having the scripts deployed as part of the install, so in combination with the log files, you can see exactly what's been done to your environment and modify if necessary.

Extensions

With native cmdlets and community scripts, together with WMI, XML and .NET code, you can achieve any desired functionality with PowerShell scripts, and have them rapidly developed and tested. So you can easily add code to update version numbers in config files, remove your assemblies from the GAC on uninstall, access performance counters etc. And PowerShell scripts are just plain text so you can extract them into a T4 template and generate different scripts for different environments in your build process.

Managing Concurrency over Service Boundaries

EltonStoneman — Wed, 15 Apr 2009 22:15:04 GMT

[Source: http://geekswithblogs.net/EltonStoneman]

Managing concurrency within an application boundary can be straightforward where you own the database schema and the application's data representation. By adding an incrementing lock sequence to tables and holding the current sequence in entity objects, you can implement optimistic locking at the database level without a significant performance hit. At the service level, the situation is more complicated. Even where the database schema can be extended, you wouldn't want the internals of concurrency management to be exposed in service contracts, so the lock sequence approach isn't suitable.

An alternative pattern is to compute a data signature representing the retrieved state of an entity at the service level, and flow the signature alongside the entity in Get services. On Update calls, the original data signature is passed back and compared to the current signature of the data; if they differ then there's been a concurrency violation and the update fails. The signature can be passed as a SOAP header across the wire so it's not part of the contract and the optimistic locking strategy is transparent to consumers.

The level of transparency will depend on the consumer, as it needs to retrieve the signature from the Get call, retain it, and pass it back on the Update call. In WCF the DataContract versioning mechanism can be used to extract the signature from the header and retain it in the ExtensionData property of IExtensibleDataObject. The contents of the ExtensionData property are not directly accessible, so if the same DataContract is used on the Get and the Update, and the signature management is done through WCF extension points, then concurrency control is transparent to users.

I've worked through a WCF implementation for this pattern on MSDN Code Gallery here: Optimistic Locking over WCF. The sample uses a WCF behavior on the server side to compute a data signature (as a hash of the serializable object – generating a deterministic GUID from the XML string) and adds it to outgoing message headers for all services which return a DataContract object. On the consumer side, a parallel behaviour extracts the data signature from the header and adds it to ExtensionData, by appending it to the XML payload and using the standard DataContractSerializer to extract it.

The update service checks the data signature passed in the call with the current signature of the object and throws a known FaultException if there's been a concurrency violation, which the WCF client can catch and react to:

Sixeyed.OptimisticLockingSample

The sample solution consists of four projects providing a SQL database for Customer entities, WCF Get and Update services, a WCF client and the ServiceModel library which contains the data signature behaviors. DataSignatureServiceBehavior adds a dispatch message formatter to each service operation, which computes the hash for any DataContract objects being returned, and adds it to the message headers. DataSignatureEndpointBehavior on the client adds a client message formatter to each endpoint operation, which extracts the hash from incoming calls, stores it in ExtensionData and adds it back to the header on outgoing calls.

Concurrency checking is done on the server side in the Update call, by comparing the given data signature to the signature from the current object state:

Guid dataSignature = DataSignature.Current;

if (dataSignature == Guid.Empty)

{

//this is an update method, so no data signature to

//compare against is an exception:

throw new FaultException<NoDataSignature>(new NoDataSignature());

}

Customer currentState = CustomerEntityService.Load(customer);

Guid currentDataSignature = DataSignature.Sign(currentState);

//if the data signatures match then update:

if (currentDataSignature == dataSignature)

{

CustomerEntityService.Update(customer);

}

else

{

//otherwise, throw concurrency violation exception:

throw new FaultException<ConcurrencyViolation>(new ConcurrencyViolation());

}

A limitation of the sample is the use of IExtensibleDataObject to store the data signature at the client side. Although this is fully functional and allows a completely generic solution, it relies on reflection to extract the data signature and add it to the message headers for the update call, which is a brittle option. Where you have greater control over the client, you can use a custom solution which will be more suitable – e.g. creating and implementing an IDataSignedEntity interface, or if consuming the services in BizTalk, by using context properties.