Smart Client

Emulating multi-touch device for Windows 7

Szymon Kobalczyk — Wed, 11 Mar 2009 03:35:19 GMT

Short while after I wrote about resources for multi-touch on Windows 7 Daniel D left a comment that got me very excited:

MultiTouchVista now has a driver that emulates multitouch hardware for Windows 7

I got to try it out myself! Bit later I got everything up and running and I’m happy to report that it’s all true: you can now effectively emulate multi-touch devices under Windows 7. You can see it yourself on this video. In fact it works so well that I was able to demonstrate it last Saturday at 4Developers conference.

That being said, the process to get it running involves few steps so I thought it would help to have a walkthrough to follow for anyone who would like to try it. So here it goes.

Hardware

Of course you need a PC. Note that computer vision puts CPU to heavy use, and I was able to get barely 15 FPS on my laptop with 1.6Ghz, and thus the interaction wasn’t very smooth. In fact Windows 7 specification requires multi-touch devices to report at least 50hz per finger.

The software that we are going to use allows to work with number of protocols. One option is to simply connect multiple USB mice to emulate multi-touch. But it is much more fun to build your own surface-like table. For start you can built a MTmini table designed by Seth Sanders – it takes about 20 minutes to build and all you will need is:

Webcam – people try different kinds of webcams for this, but I think even something simple with decent video should work. I have great results using Logitech QuickCam Pro 9000 because it has excellent video quality in 640x480 and most important you can set the focus and other parameters manually.
Tracing paper – it’s purpose is to diffuse the light coming to the camera so only objects very close to the surface (i.e. your fingers) look sharp. Initially I used just ordinary printer paper, but tracing paper gives more even image.
Photo frame – no, I’m not talking about any fancy LCD frames. It’s just plain wood frame (word of advice: bad things will happen if you try to repurpose frame from your wedding photo). Frame I use has A4 paper format (210mm × 297mm) and fits nicely on top of the carton box from the packs of A4 office paper.
Cardboard box, duck tape, and scissors.

When you have all the materials here is what you do:

Put the camera at the bottom of the box facing up, and glue it with the duck tape. You might need to cut out the hole for the camera’s USB cord.
Remove the back board from the photo frame and put only the tracing paper on the glass.
Put the frame on top of the box.

You can also watch this video for step by step instructions.

Here is my completed setup:

Software

Obviously you need to have Windows 7 Beta installed. So hopefully you was able to download it while it lasted.

Next thing to download and configure is tBeta. Here is description of it from official website:

The Beta, tbeta for short, is a open source/cross-platform solution for computer vision and multi-touch sensing. It takes an video input stream and outputs tracking data (e.g. coordinates and blob size) and touch events (e.g. finger down, moved and released) that are used in building multi-touch applications. tbeta can interface with various web cameras and video devices as well as connect to various TUIO/OSC enabled applications and supports many multi-touch lighting techniques including: FTIR, DI, DSI, and LLP with expansion planned for the future (custom modules/filters).

Download the software from the above website and unzip to folder you choose. Before you run it you might want to change configuration to use higher resolution if your webcam supports it. Go to \tbeta\data folder and open config.xml file. Specify the correct values for WIDTH and HEIGHT in CAMERA_0 node.

One more thing I did to get more predictable results was to turn off all automatic adjustments in camera software. For Logitech QuickCam this includes switching to manual focus, and turning off Automatic settings for RightLight, exposure and gain. We are not going to move the camera around or dramatically change the lightning conditions so it will be best to make all settings constant.

When you run tbeta.exe you should see something like this:

The tricky part is calibration, which means adjusting the sliders for all filters at the bottom so that you get best recognition accuracy. Unfortunately I don’t know any rules that would help you with this. You would have to play with the settings a bit to get a feeling how it works. If you make it right you should see an outline and unique id associated with each of your fingers as seen on the above screenshot. When all works well you can minimize this window (press spacebar) so it won’t consume CPU for rendering.

The last piece of this puzzle is MultiTouchVista project that I mentioned in previous post as well. This framework, which actively developed by Daniel D himself (aka nesher), adds support of multi-touch to the current version of WPF. But for purpose of this article the important thing is the recently added Windows 7 multi-touch driver. It’s not even officially released yet, so you would need to go directly to Source Code tab and download latest bits (I run it on changeset 18685).

You can follow instruction in this video on how to install the driver and run the services. Here is a short summary of the same:

Compile all projects for MultiTouchVista. Follow instructions on this page.
From Windows Explorer go to MultiTouchVista\Main Source folder, and while holding down Shift right click on Multitouch.Diver folder and select “Open Command Window Here”.
In this window type: install driver.cmd, press Enter and ignore all warnings about not certified drivers.
Open Device Manager and confirm that new driver called “Universal Software HID device” is installed under “Human Interface Devices”. Video suggests to disable and then enable the driver to ensure it is working correctly.
To verify the driver is installed you can open “Pen and Touch” applet from Control Panel. It should now contain the Panning tab.
Back in Explorer go to MultiTouchVista\Main Source\Output folder and run Multitouch.Service.Consol.exe. By default it loads the MultipleMiceInputProvider so if you should see red dots indicating “virtual cursor” for each USB mouse you have attached. But to use your MTmini table you would need to change configuration so it connects to tBeta through TUIO interface.
From MultiTouchVista\Main Source\Output run Multitouch.Configuration.WPF.exe. From list of Available devices select Tuio and click the arrow button in middle to make it Active device. Then click “Restart service” button to apply the new configuration.
Finally go to MultiTouchVista\Main Source\Multitouch.Driver.Console\bin\debug and run Multitouch.Driver.Console.exe.
Now if you put finger on surface, you should see the input messages coming to the console window. And of course now you should be read to use it as input device for Windows.

Try it on several touch enable applications that I listed in my last post (XPS Viewer and Paint are great for quick demo). You can also find some C++/C# examples on MSDN Code Galery.

Great thanks for Daniel D for leaving this inspiring comment, and even bigger compliments for developing such great library. Please don’t hesitate to let me know if I can help with anything.

Resources for Multi-touch in Windows 7

Szymon Kobalczyk — Fri, 13 Feb 2009 14:19:14 GMT

Last couple of months I’ve been working on very cool project that utilizes new multi-touch features in Windows 7. Although I can’t talk yet about our product, I thought it would be good to start sharing my experience with multi-touch programming. I’ll begin with some general resources to get you started.

Prerequisites

Of course first thing you need is a multi-touch capable hardware. As far as I know currently there are only three devices available on the market. Those lucky to be at PDC could see that most demos were run on HP TouchSmart All-in-One PC, or Dell Letitude XT tablet. Later in December HP released TouchSmart tx2z tablet, and this week Dell announced specs on Letitude XT2. For my work I use HP TouchSmart IQ504 PC.

The good news is that we have already seen some announcements from other manufacturers, and can expect number of devices to increase closer to Windows 7 release date.

In terms of touch-screen technology both tablets use the same DuoSense capacitive digitizer from N-Trig, while HP All-in-one PC uses optical overlay developed by NextWindow. The main difference is that NextWindow device supports only two touch points, while N-Trig’s can recognize more contacts.

Unfortunately multi-touch works only with dedicated hardware, so you can’t use other digitizers (like Wacom’s), touchpads or TabletPCs. Also there is no way to emulate multi-touch on Windows 7, for example by attaching multiple mice, although this is supported on Surface SDK emulator as demonstrated by Scott Hanselman (see around 14:30).

Software

In terms of software you need two things: Windows 7 and proper multi-touch driver for your device. Currently I run on Windows 7 Beta 1 (build 7000 for x86), and when I did clean install today most of the drivers are now available through Windows Update. The only driver I had to install manually was for Ralink WLAN adapter. However the TouchSmart version I have doesn’t have TV tuner, so if yours does you might need to install some additional software. Fortunately Kurt Brockett published detailed guide how to setup Windows 7 on HP TouchSmart.

If by any chance the multi-touch drivers won’t install for you automatically, you can download them directly from NextWindow’s or N-trig’s websites:

NextWindow’s Multi-Touch Screen Driver (Current version is 1.0.3.8 – the same as on Windows Update)
N-trig’s Multi-Touch download for Win7

To verify all works fine open any page in IE8 and you should be able to use two fingers for zooming.

Here are some other areas in Windows 7 that were enhanced with multi-touch features:

Panning with inertia is enabled “everywhere” where scrollbars exist.
Paint
- Choose a brush from the “brush gallery” and you can then multi touch finger paint.
Games
- Hearts/Solitaire have been optimized for touch
Shell
- Windows Snapping (Aero Snap) with Touch
- Aero Peek with Touch
- Taskbar Jump Lists with Increased spacing
Windows Media Player
Windows Photo Viewer
- Zoom, Rotate, Panning and Flicks
XPS Viewer
- Gestures (Zoom, Two-Finger Tap, Panning)
Media Center
- Direct Panning in most Scrollable Views and Menus
Touch Keyboard / TIP
- Multi-touch touch keyboard
Internet Explorer 8
- Panning
- Drag Menu (Address Bar) with Increased Spacing
- Increased Spacing for Favorites with Touch
- Gesture (Zoom, Panning, Flicks back and forth)
Windows Live Photo Gallery
- Gesture support in the Viewer

You can also download IdentityMine’s Air Hockey game from: www.identitymine.com/airhockey

Tip: If you are using touch on regular basis touch it is a good idea to increase the size of Windows UI (fonts, icons, etc.) by changing your display settings to Medium (125%). This will make Windows much more “touchable”.

Multi-touch Programming

Most of what we know so far about multi-touch API’s introduced in Windows 7 and WPF 4.0 comes from PDC session by Reed Townsend and Anson Tsao.

There are also two later sessions from WinHEC 2008: Multi-Touch in Windows 7 Overview covers some basics and repeats information from PDC, while Multi-Touch Driver Development and Logo Compliance is more toward hardware developers.

Recently published Windows 7 SDK Beta includes header files for WM_TOUCH, WM_GESTURE messages, and related functions and structures. It also includes some documentation and samples on this (also available on MSDN).

When you have all of above installed, you can try some code samples from hands on labs from PDC available on MSDN Code Gallery. These samples are both in C++ and C#. But to get it working in WPF you will need some more advanced framework in place, and I suggest you first take a look at the Multi-touch Vista project on CodePlex. You can take a look at MIRIA SDK that adds multi-touch support to Silverlight apps. Finally, for those of you that already work with TUIO libraries (like Touchlib) here is an applet to translate WM_TOUCH to TUIO messages.

More information in multi-touch and related technology can be found in the great NUI Group Forum. You can also find all information how to build multi-touch device yourself.

I hope that this information will get you started, and I will try to publish more information on multi-touch programming in C# in next few days. If you have any questions or suggestions please leave a comment.

Two cool Microsoft Surface apps from InterKnowlogy

Szymon Kobalczyk — Fri, 29 Aug 2008 13:41:02 GMT

InterKnowlogy is always on the cutting edge of the Microsoft Platform so its not surprise that we were among first to build applications in WPF, Silverlight and now on Microsoft Surface (a touch screen computer embedded in a coffee table). I'm excited to show you two applications that we released recently.

VitruView

VirtuView is a collaborative environment for viewing and annotating patient's examination results on anatomic 3D models. This application is evolution of our earlier Angiographer that we build for InterMountain Healthcare together with our partner Zygote (world's leader in 3D human anatomy models and textures). VirtuView lets users zoom and rotate the 3D heart models with your hands. You can use your fingers to draw arteries, place Stents and add annotations. And because this is multitouch you can use more than one finger or many people can work at once. This application is done in WPF and uses Microsoft HealthVault to securely store patient's data.

Check out this video demonstration from Tim Huckaby and Kevin Kennedy:

Update: Here you can watch another video with Tim and Dr. Peter Kuhn from Health User Group Conference.

History at your fingertips

This amazing application was built in conjunction with Microsoft and the Library of Congress. It chronicles the history of the republican and democratic national conventions in interactive Surface experience. The application is featured LIVE on Surface units at both upcoming conventions, and it was shown on the ABC National News this week and you can watch it on the ABCnews.com website (move forward to approximately 01:18).

Here is a more technical demonstration from Rodney Guzman:

What about me?

Meanwhile I've been working with great team at Colonial Life & Accident Insurance Company that built the Harmony solution. We helped them built a WPF offline version of the Web based Harmony self-enrollment system that will be used by insurance agents. You can find more details in this Microsoft case study.

How I won the Lab49 WPF in Finance Innovation Contest?

Szymon Kobalczyk — Tue, 18 Mar 2008 02:39:06 GMT

Honestly, I still don't know how I did it. The LAB49 WPF in Finance Innovation Contest was announced back in December and I think I first read about it on Tim Sneath blog. With all the cool prizes I was very inclined to participate, but it quickly turned out that its available only to US citizens. However that changed in the first week of February, so I started considering it again, but still didn't had any clue what to do. You see the goal of this contest was to create a WPF application that visualizes a set of provided financial data in some interesting way. While the first part was easy (writing the app in WPF) the hardest part was to figure out what to do with the data. Only after "last call to action" email from Daniel Chait I decided its about time to start coding.

Because I started working late on this project, initially I wanted to create only a charting control capable of displaying the line and candlestick plot of stock prices. But after the contest deadline was extended I started thinking of a better way to visualize the stock prices for a given day – something that goes beyond a simple listbox or a datagrid. The next option I considered was to create a heatmap – i.e. use color gradients to denote the change in stock prices relative to the previous day. But in all samples I’ve seen these symbols were placed on a regular grid and the placement didn’t correspond in any particular way with the data. I started thinking about how to sort the symbols so that those that experienced similar data change would be close to each other, and those with different data change remain further apart. This finally led me to the idea of applying Craig Reynolds’ flocking algorithm to perform this clustering.

Here is a screenshot from my final entry called Stock Information Boids:

Also you can already download the source code for this application from my resource page at MSDN Code Gallery. Make sure to read the user guide that describes how the algorithm works and how to use the application.

Turns out that for some reason the judges liked my solution, because last week at the closing keynote of 2008 Microsoft Financial Services Developer Conference in New York, Daniel Chait announced that I won the grand prize in the contest!

The two finalist were Jacob Carpenter and Jobi K Joy, while the honorable mention went to Paul Hounshell. Congratulations guys, great work! You can download their very cool applications from here:

In following weeks I will try to share some details on my implementations (in particular about the Timeline and Flock controls). I was already told that Jobi K Joy and Jacob Carpenter plan to do the same, so make sure to subscribe to their blogs too.

I would like to thank the judges for selecting my entry. And last but not least, big kisses for my wife Joanna for letting me work on the project all nights and weekends considering her present condition (L)

Introducing WikiSpider

Szymon Kobalczyk — Thu, 31 Jan 2008 04:22:47 GMT

About two weeks ago Daniel Biesiada (who is ISV DE here in Poland) announced on his blog a little programming contest. The goal was to build a .NET application that would check if the the theory of Six Degrees of separation applies to two given topics in Wikipedia. In order words to find a path from the source page to destination with no more then six links. At the time I had not much else to do (apart from setting up website for the C2C Conference, helping out with the European Silverlight Challenge, and preparing for the WPF Beta Exam) so I decided to give it a try.

Fast forward two weeks and I present you my WikiSpider:

As usual building this took me much more time than I initially anticipated (including few sleepless nights). And still I didn't make it before the deadline, so this even didn't count as a contest entry anymore (sigh!). However this was mainly because my personal goal was to throw in there every new piece of .NET 3.5 I could find fit - and most of them I never used before.

Here are some key technologies I managed to put into this:

The UI is done in WPF (and this was the only thing here I knew a bit about). However I borrowed the graph control from the excellent Kevin's WPF Bag-o-Tricks.
The caching is done using SQL Server Express. Initially I wanted to do this using SQL Compact but I run into performance issues and had to switch to full SQL in order to run the queries in profiler. But since this was fixed (with big help from Paweł Potasiński) I could try with SQL Compact again.
Of course data-access is done using LINQ to SQL. And of course this was the main source of my problems, as it was first time I've done anything in it, and so far I only read the Scott Gu's tutorials. Still, I'm already in love with it.
Speaking of LINQ. Initially we were screen scrapping the HTML pages to get all the links. But turns out that Wikipedia has a little known about Query API that enables to get the page content in XML. So the obvious move was to rewrite this part with LINQ to XML.
The path-finding algorithm was borrowed from Eric Lippert. The nice thing about it is that it uses lots of C# 3.0 language features, so it is a great resource to learn from. The new C# syntax is so addictive that I already miss it in my other project.
Finally, I wanted to publish the app with ClickOnce but run out of time. So maybe later.

I learned many interesting things and tried out some new stuff that I wanted to check out anyway. I will try to share my discoveries in the next few days, but in the meantime feel free to download and take a look at may code (I know it's not prettiest piece of code you've seen but I was in a rush to finish this on time):

Download the source code

Here you can also download the entries from other participants: Łukasz Sowa, Maciej Rutkowski, and Arkadiusz Benedykt. Congratulations to all of you!

Installation

Download the code from the above link and extract it.
The application uses local SQL database for caching and unfortunately you need to create it yourself (now you know why I wanted to use SQL Compact). Simply launch SSMS and create empty database called WikiCache.
Run the Create_WikiCacheDB.sql script from the data folder to create the database schema.
By default the app is configured to look for the WikiCache database on the local SQLEXPRESS instance. If you installed it somewhere else update the connection string in app.config accordingly.
Run the build.bat or open solution in Visual Studio 2008 and run from there.

Usage

Enter the name of the Wikipedia page in the address bar at the top and press the Go! button. The entered topic and the pages it links to will be displayed as graph.
Clicking on any topic will make it currently selected (put it in the center of the graph).
Right-click on any topic to open the context menu. Select "Open in browser" to.... load the page in browser.
Select "Set as source" or "Set as destination" to put the topic name in appropriate field on the sidebar
[Note: Currently it's the only way to show the sidebar]
You can also enter the source/destination topics manually.
When both are set click on the Start button to begin searching for the path. Few statistics are displayed on the bottom of the sidebar.
During the search you can still use the graph or navigate to other pages (thanks to the BackgroundWorker magic).
When path is found it is displayed on the sidebar, and you can click on each topic to center it on graph.

Have fun!

White paper: Designing Smart Clients Based on CAB and SCSF

Szymon Kobalczyk — Sat, 13 Jan 2007 17:09:00 GMT

A year ago I've published a series of articles on understanding Composite UI Application Block (CAB). Although I've never got to finish it I still occasionally receive comments and questions on this subject.

Recently I came across two great papers on CAB and SCSF that I would like to share:

Designing Smart Clients Based on CAB and SCSF by Mario Spuszta, is architectural guidance created for RACON Software GmbH, and later published as white paper on MSDN. It contains complete overview of CAB and SCSF, with many interesting tips on implementation and project management. It answers many question I had when I was learning CAB. What I particularly liked is the use case-driven strategy for identifying work items.
Commonwealth Bank of Australia CommSee Solution: Case Study. Commonwealth Bank of Australia designed, developed, and implemented its CommSee application, a relationship banking solution custom-built by the Commonwealth Bank of Australia. This project was major success and has been successfully deployed to 30,000 users at more than 1,700 sites across Australia. Its architecture inspired patterns&practices implementation of CAB and serves as great source of ideas on building composite smart clients and SOA applications. Ron Jacob recorded series of video interviews with the CommSee core team available on Channel 9.

Understanding Composite UI Application Block, Part VII

Szymon Kobalczyk — Sun, 26 Feb 2006 13:57:00 GMT

There seams to be some confusion around WorkItem's State. I've seen few posts on CAB message boards with questions if it's OK to put certain data into it. I will try to add my two cents to it but note that this is my point of view and it might be wrong.

Let's think for a minute what kinds of data we can have in our applications. First that comes to mind is the data that is presented to the end-user and this of course must be stored in the View (SmartPart) for the time it's displayed. This data can be loaded from some external sources (files, database) but this should be handled by services. To do their work services can also carry some runtime data that is not visible to the end-user.

So we have some flow of data between Views and Services (preferably with Controller or Presenter in the middle). But many times we also need to share same data between several Views. This will be for example in classical master-detail scenario when one View presents list of records and the other one shows details of currently selected record. Clearly the information of selected item lays outside of the service's domain because the same service can be used by several Views so from this standpoint it should be “stateless“. So here is where WorkItem's state comes in handy. In short State property is a loosely typed collection of items identified by some key. In my understanding it should be primary used for communication between SmartParts and other Items contained within the same WorkItem and its child WorkItems.

There are several mechanisms in CAB that help us with this scenarios.

State change notification

Often we want to react somehow to changes in State item's value. CAB automatically publishes events to state changes and allows to declare handlers simply by putting StateChangedAttribute on a method:

[StateChanged("MyProperty")]
public void MyProperty_StateChanged(object sender, StateChangedEventArgs args)
{
 // handle state change
}

The StateChangedEventArgs contain State item's key, and both old and new values. Of course there can be more then one handler to this event. Also check out John Luif's blog entry on this.

Injecting State into child WorkItems

Sometimes we need to pass the current value of some State item from parent WorkItem to the newly created child WorkItem. We can do this by manually copying the respective values or let the CAB do the work for us. To indicate that property value or constructor parameter should be injected from parent WorkItem's State we can use the StateAttribute:

[State("MyProperty")]
public string MyChildProperty
{
get { ... }
set { ... }
}

Note that this will only copy the value when child WorkItem is created and won't update it when it changes on parent WorkItem. There are some restrictions when using state injection and be sure to read two excellent entries on Mariano's Szklanny blog on how to avoid them.

Persisting WorkItem's State

Finally, CAB defines IStatePersistenceService that allows to persist WorkItem's State between sessions. Out of the box CAB provides to implementations: FileStatePersistenceService and IsolatedStorageStatePersistenceService. Both store state in binary files using the BinaryFormater serializer, but the later one uses Isolated Storage so it should work in restricted access scenarios. Both inherit from StreamStatePersistantService base class with you can also use to implement you own service.

This service does not load by default and you need to add following entry in app.config to enable it:

<CompositeUI>
 <services>
 <add serviceType="Microsoft.Practices.CompositeUI.Services.IStatePersistenceService,
Microsoft.Practices.CompositeUI" 
 instanceType="Microsoft.Practices.CompositeUI.Services.IsolatedStorageStatePersistenceService,
Microsoft.Practices.CompositeUI" />
 span class="html">services>
span class="html">CompositeUI>

To store or retrieve WorkItem's State simply use it's Save() and Load() methods. Note that it uses WorkItem's name to uniquely identify that WorkItem's state so make sure you assign it to the same value before saving and loading.

I haven't tried this one yet but it opens some interesting possibilities.

As example of using state let's take a look at the Inventory WorkItem of my sample application. There are three primary views in this WorkItem: Product list, Items list and Item details. When user selects a product category the first view shows list of products for this category. Next when product is selected second view shows product items. Then after selecting the item it's details are presented on the final view. The IDs of currently selected records are stored in Inventory WorkItem's state and the View's are wired to state change events so they react immediately to any changes.

Currently only one view is displayed at a time but now we can easily build an "Advanced View" that will show all three views at once. We will start by creating container SmartParts for both modes. SimpleView hosts a DeckWorkspace while AdvancedView contains ZoneWorkspace with two SplitContainers. Both also contain simple link buttons to switch the view. These buttons invoke a TogleValueCommand that changes the value of the ViewMode property. This in turn rises a StateChange event that is handled in ViewMode_StateChanged method. Finally this method changes the current container view and moves the three inventory views to current workspace. It also tries to maintain current active SmartPart.

You can see the inventory screens in both modes below:

But the nicest part in all this that I didn't have to change the existing SmartParts at all! They were already communicating with State and changing their arrangement on screen didn't required any changes to their code. Isn't that cool or what :-)

Updated 10/05/2009: Since ProjectDistributor.net is gone I’ve moved the source code to my SkyDrive. You can grab it here:

CAB sample published and future plans

Szymon Kobalczyk — Sun, 19 Feb 2006 17:46:00 GMT

The source code for SmartPetShop, my sample project for Composite UI Application Block, is now available for download.

If you want to run it, first I recommend to download and install the original PetShop.NET 3.2 from Microsoft , as it will setup properly the database and register COM+ services. After that you should update the connection strings in Win application's app.config with encrypted settings taken from original Web PetShop's web.config. Simply find settings like and copy them to app.config. Later I will try to provide my own installation scripts for this. Finally, to compile the code you would need to place CAB DLL's om the Lib folder.

Currently my changes concentrate only on presentation layer so I simply replaced the Web project with my own Win project. All other code remains unchanged (with kind of proves that this is indeed well architected application).

Future plans

My first goal was to create sample application that would demonstrate how CAB can be used in real-life project. But during the process I realized I can further extend this sample and try to apply some other new technologies emerging in .NET world. So here are my future plans for this project.

Finish conversion to Windows Forms
From the original Web PetShop there is only couple of functions left to implement (for placing orders). But looking at CAB I already covered following features:
To make it complete I still need to discuss:
- WorkItem's State and StateChange events
- Services
- Events and background processing
- Modules and WorkItem extensions
- ObjectBuilder
Reimplement Data Access Layer using Enterprise Library
CAB is only a little brother of EntLib so I think it makes sense to show how to use them both in the same application. I'm also curious to what degree it will simplify the original code for the data access and business logic layers. I will definitely explore the Data Access AppBlock but I will also try include others like Logging, Exception Handling and Caching. Also, I should try to replace the PetShop's ConfigTool with EntLib's Configuration. At the same time I would try to deploy the database on MS SQL 2005 Express using user instance databases and embedded update scripts (seen nice articles on both recently).
Separate client code from server
I called my version SmartPetShop to indicate that it will support the SmartClient architecture. There are three main characteristics of SmartClients: rich user experience, intelligent updates, and offline capabilities. If we assume that my CAB implementation provides richer experience that the previous Web based (actually it tries to mimic it closely) there are two things left to address. For installation and updates I would use ClickOnce, but there is not much to say that wasn't well known already. So last thing is to enable the application to work without connection to server. In this mode it should at least enable to browse the product catalog and place items in shopping cart. Then, when application goes on-line, user can submit orders and check status of earlier orders.

Currently the application connects directly to database so the main task here would be to move the data access and business logic to separate server process. Necessity to connect these two will allow me to explore another exciting new technology, the Windows Communication Foundation (aka Indigo). This will also test the current architecture as it seems that it should suffice to switch ServiceAgent implementations and there should be no change to UI logic.
Another type of client
When I'm done with above I should have working example of SmartClient. Then I can try to implement another type of clients. There are two candidates. I'm really excited to start working with Windows Presentation Foundation (aka Avalon), however I don't think I would have opportunity to apply it in my professional work in foreseeable future. It is more likely that I would need to refresh my skills in ASP.NET. I stopped working on Web apps almost one and a half years ago and didn't had opportunity to try out all the goodies in ASP.NET 2.0. I'm also excited about the Atlas extensions since I've done some work with AJAX technologies on my own (some really cool DHTML behaviors).

So here you have it. I will try to do as much as I can of this in upcoming months. I need to learn these things anyway but if I also put myself to write about it the process is more systematic. Feel free to drop me a note if you have additional ideas or like me to explore any particular topic more closely.

Update: I just discovered that last week Microsoft released .NET Pet Shop 4.0. What a perfect timing! I guess I need to revise my goals as some became obsolete. Most notably these guys saved me work (and pleasure) of upgrading to ASP.NET 2.0. I haven't reviewed the code yet but judging from the accompyning article most changes were made to the presentation layer (bummer!). However, it seems it would be easier to update my code base now before starting any modifications to other layers. Anyway, it's great this project is not dead and I'm sure it will be valuable source of information on building enterprise .NET 2.0 applications for everyone.

Understanding Composite UI Application Block, Part VI

Szymon Kobalczyk — Mon, 06 Feb 2006 01:54:00 GMT

It's been over a week since my last post on Composite UI App Block, but don't start thinking I have given up on it already. I still want to finish the PetShop sample and at the same time to cover all important parts of CAB on this blog.

Last time I have covered some basics on using Commands and UI Elements and demonstrated how I implemented commands in my sample application. But as I said before, out of the box CAB only has UIElementAdapters for ToolStrips and MenuStrips. Just to get started I put all my commands on the main menu, but the original PetShop was a web app so this were all links. So it got me thinking what it takes to implement my own UIElementAdapter. For most people support for menus and toolstrip could be enough, but some of us use third-party controls or wants to invoke commands from other controls like trees, or Outlook style navigation bars. I think knowing how to extend CAB to handle such situation can be helpful.

(On the other hand, I heard on ARCast that Infragistics committed to release CAB wrappers for their controls so those of you who use NetAdvantage don't need to read any further).

First thing was to create the control that will display my links and fire events when one is clicked. For this I have extended the standard LinkLabel adding collection of my own LinkListItems. These items are similar to a menu items having Text, Enabled, and Visible properties, and a Click event. The control automatically updates itself based on these settings. Under the hood it constructs a single line from the texts of all visible items inserting bars ( | ) as separators, and it positions a LinkArea on each enabled item. Then upon receiving LinkClicked event it propagates it back to corresponding item and fires its Click event. It took ~200 lines of code but there is nothing fancy so I won't bore you with it here.

Now having the control ready I need to plug it into the CAB framework. There are three additional elements that I need to provide: IUIElementAdapter, IUIElementAdapterFactory, and CommandAdapter.

As explained in last post, UIElement adapter is responsible for properly showing items of given UIElement type. This interface has only two methods: Add and Remove that are called each time items are added or removed from extension site. In my case these methods simply add or remove items from the LinkList items collection:

public class LinkListUIAdapter : UIElementAdapter
{
    LinkList linkList;

    public LinkListUIAdapter(LinkList linkList)
    {
        Guard.ArgumentNotNull(linkList, "linkList");
        this.linkList = linkList;
    }

    protected override LinkListItem Add(LinkListItem uiElement)
    {
        if (linkList == null)
            throw new InvalidOperationException();

        linkList.Items.Add(uiElement);
        return uiElement;
    }

    protected override void Remove(LinkListItem uiElement)
    {
        if (uiElement.Owner != null)
            uiElement.Owner.Items.Remove(uiElement);
    }
}

The role of IUIElementAdapterFactory is to provide appropriate UIElementAdapter instances for given UIElement's type. The Supports method should indicate if particular factory supports the given UI Elements type, and then GetAdapter methods should create adapter for that UI Element.

public class LinkListItemAdapterFactory : IUIElementAdapterFactory
{
    public IUIElementAdapter GetAdapter(object uiElement)
    {
        if (uiElement is LinkList)
            return new LinkListUIAdapter((LinkList)uiElement);

        throw new ArgumentException("uiElement");
    }

    public bool Supports(object uiElement)
    {
        return (uiElement is LinkList);
    }
}

If we want to be able to attach command to the UI Elements we also need to provide a CommandAdapter. It's purpose is to provide logic that is needed for UIElement to invoke a Command, and also to synchronize changes to Command with the UI Element. Luckily for me the first part is handled by the base class EventCommandAdapter, so I only had to make sure that changes to Command status will be applied to the corresponding link:

public class LinkListItemCommandAdapter : EventCommandAdapter
{
    public LinkListItemCommandAdapter()
        : base()
    { }

    public LinkListItemCommandAdapter(LinkListItem item, string eventName)
        : base(item, eventName)
    { }

    protected override void OnCommandChanged(Command command)
    {
        base.OnCommandChanged(command);

        foreach (KeyValuePairstring>> pair in Invokers)
        {
            pair.Key.Enabled = (command.Status == CommandStatus.Enabled);
            pair.Key.Visible = (command.Status != CommandStatus.Unavailable);
        }
    }
}

Having all these three pieces we can now register them in CAB. It should be done in the AfterShellCreated method of the ShellApplication class:

protected override void AfterShellCreated()
{
    base.AfterShellCreated();

    ICommandAdapterMapService mapService = 
        RootWorkItem.Services.Get();
    mapService.Register(typeof(LinkListItem), typeof(LinkListItemCommandAdapter));

    IUIElementAdapterFactoryCatalog catalog = 
        RootWorkItem.Services.Get();
    catalog.RegisterFactory(new LinkListItemAdapterFactory());
}

Finally we have all set up and we are ready to use the new UI Elements. But the best thing it was, that the only change I had to make was to replace references to ToolStripMenuItem with my own LinkListItems like this:

  LinkListItem signInItem = new LinkListItem("SIGN IN");
  UIExtensionSites["NavigationBar"].Add(signInItem);
  Commands["SignIn"].AddInvoker(signInItem, "Click");

All the other code that handles commands and updates their status remains unchanged. And thats the real advantage of CAB!

Understanding Composite UI Application Block, Part V

Szymon Kobalczyk — Wed, 25 Jan 2006 00:28:00 GMT

In most applications certain user interface areas are shared among various modules. Examples are all kinds of menus, tool bars, status bars, Outlook style sidebars and others. What is common for all of them is that they serve as containers for collections items (menu items, toolbar buttons, etc.) that user can activate to trigger some actions in the application. For example, the File > Open menu item normally would invoke the action of opening a file. However same action could be also accessible using a toolbar button. So it is desirable that we could decouple the user interface elements from the actions that they trigger. CAB provides solution for this in the form of UIElements and Commands.

UIExtensionSites are the shared containers that can contain your UIElements. Out of the box CAB provides support for MenuStrips and ToolBarStrips but you can easily add support for other elements by implementing your own UIElementAdapter. After you add a menu or toolbar to your SmartPart you need to register it as an extension site, by calling the RegisterSite method on the WorkItem.UIExtensionsSites collection and providing unique identifier for that site. Usually extension sites are registered in the RootWorkItem by the developer of shell the application. Later when module developer needs to add his own items to that site he only need to create these items as usual and than call the Add method of the specific UIExtensionSite. The corresponding UIElementAdapter is responsible for properly showing items for given UIElement type.

Next step is to attach Commands to your UIElements. Commands are used to assign easily the same code to multiple UIElements. To declare command you need to create event handler that implements your action. Then you add the CommandHandler attribute to the method declaration specifying command name. To assign Command with a UIElement you can call the AddInvoker method on specific Commands collection passing reference to the UIElement and specifying name of its event that will trigger the action.

All pages in the .NET PetShop sample share a common NavigationBar that contains links to following pages: Sign In, Sign Out, Edit Account, View Shopping Cart and the Search TextBox. So we can consider them as UIElements that invoke Commands in different modules. For example, SignIn command is handled in the Account module, and ViewShoppingCart is handled in the Orders module. So it would be nice if each module could register the commands it provides and the Shell doesn't need to know anything besides providing an extension site for them.

Because out of the box CAB only supports UIElementAdapters for toolbars and menus for now I will implement the navigation bar in form of the main menu to see how this works. Later I can implement a custom UIElementAdapter that would closely match the web style.

First I need to add a MenuStrip to the the ShellForm and make it publicly accessible. Then I can register it as UIExtensionSite by adding following line in ShellApplication's AfterShellCreated method:

  RootWorkItem.UIExtensionSites.RegisterSite("NavBar", Shell.menuStrip);

Basically that is all that shell developer needs to do -- the rest belongs to module developers. So I quickly switch hats, and go to the account management module. Let's start by adding the SignIn button. In the OnRunStarted method of AccountWorkItem I add following lines:

  ToolStripMenuItem signInItem = new ToolStripMenuItem("SIGN IN");
  UIExtensionSites["NavBar"].Add(signInItem);

As you can see, first I create a normal ToolStripMenuItem. But then, instead of adding it directly to a MenuStrip, I add it to the UIExtendsionSite defined earlier. Internally CAB finds the MenuStrip defined earlier in ShellForm and passes it together with the menu item to the correct UIElementAdapter with knows how to handle them both correctly.

Now, when I run the application I should see my first menu item and I can add items for other two commands (SignOut and EditAccount) the same way. However, clicking all these menu items does nothing so I need to attach handlers for appropriate events on these items. I could to this the usual way by directly attaching instance method to the Click event but in CAB we should use Commands to handle such things. Following line will register the Click event of the SignIn menu item as invoker for the SignInCommand:

 Commands["SignInCommand"].AddInvoker(signInItem, "Click");

To handle this command I need to declare the event handler and mark it with CommandHandler attribute:

  [CommandHandler("SignInCommand")]
  public void SignInHandler(object sender, EventArgs e)
  {
    SignIn();
  }

What is important is that command handlers and UI elements that invoke them doesn't need to be defined in the same class or in other way directly accessible. For example, the command could be invoked in one WorkItem and handled in another WorkItem, SmartPart or Presenter. And I guess that there can be multiple handlers as well as many invokers attached to single command but I haven't tried this yet.

In many cases command's availability depends of the application's state. For example, toolbar's undo button would be disabled if user hasn't made any modifications. In my case the SignIn button should be visible only if no user has authenticated and SignOut and My Account buttons should be visible otherwise. Normally I would control this by setting the Visible property on MenuItems but again in CAB I can do this indirectly using Commands. This ensures that if there are multiple UIElements for the same Command then all of them would get updated accordingly. Commands define the Status property that can have one of three values: Enabled, Disabled and Unavailable. When value of this property changes, the adapters linked to particular command can apply these changes to the corresponding UIElements. So in my case when user signs in or signs out I execute following code:

  if (userAuthenticated) 
    Commands["CommandSignIn"].Status = CommandStatus.Unavailable; 
  else
    Commands["CommandSignIn"].Status = CommandStatus.Enabled;