Willem's...

{rue if I mellow}

  Home  |   Contact  |   Syndication    |   Login
  25 Posts | 0 Stories | 129 Comments | 52 Trackbacks

News

Archives

Post Categories

Businessware Architects

XML-FX.COM

Tuesday, April 14, 2009 #

XML-FX.COM

Finally, we have found the time to be more serious about sharing our knowledge, experiences (good and bad!) in the field of XML applications and technologies. We hope this website provides useful and enlightening information and you will come back here often!

The site features a blog with (hopefully) useful articles, posts and downloads, a tools section with applications, tools and utilities, and some useful links.

Best regards, Willem


Tuesday, June 05, 2007 #

MFC provides a rather neat way of updating the state of window elements in an MFC CWnd (or derived) object during application idle-time. Typically, this is done by setting the element ID and member function in the ON_UPDATE_COMMAND_UI macro, and then declaring the function in the class header file:


    afx_msg void OnUpdateMyCommand(CCmdUI* pCmdUI);

Then you implement the handler as follows:


    void CMyClass::OnUpdateMyCommand(CCmdUI* pCmdUI)
    {
      if(IsMyCommandAvailable())
        pCmdUI->Enable(TRUE);
    }

This provides a mechanism to update the state of the UI element by passing TRUE or FALSE to the Enable function. The effect is that a toolbar button and a menu item with the same element IDs can be simultaneously updated with one state management function. Similarly, command execution can be routed through a similar mechanism, so that clicking the menu or the corresponding tool button results in a single (and the same) function being executed.

In the standard .NET System.Windows.Forms world, this functionality is not available out the box. In essence, the menu and button elements are managed separately, and require separate handlers for execution (the Click event), and no 'OnUpdate..' type handlers are defined. For moderately complex  forms, separate handlers result in many handler methods which become difficult to manage, particularly as the handlers are added to the ended of the code module in a non-structured way.

This problem can be answered in a few ways. The links below are useful resources, and provide very flexible but ultimately over-complex solutions and implementation-patterns for many applications:

The following approach provides a mechanism (read: class) to provide something similar to the command and update handlers in MFC. The Command class implemented provides the code to link together a ToolStripMenuItem menu item for use on a MenuStrip object, with a ToolStripMenuItem popup menu item for use on a ContextMenuStrip, and with a ToolStripButton toolbar button on a ToolStrip object. Furthermore, instead of linking the UI elements together with macros and element IDs, this implementation relies on correctly named elements set at design-time, which are then reflected and resolved at runtime, and linked together for the lifetime of the Command class instantiated to manage these elements together.

A Command object is instantiated and added to the static list of commands to be managed:


    Command.Commands.Add(new Command(this, "CommandA"));

The this parameter is the command target, and is normally the Form object which contains the UI elements.

The Command class expects the "CommandA" implementation to be defined in the form code in the format On{Command}:


    private void OnCommandA(object sender, EventArgs e)
    {
      // Implementation of Command A.
      ...
    }

Similarly, the "CommandA" UI update implementation is defined in the form code in the format On{Command}UpdateUI:


    private void OnCommandAUpdateUI(object sender, EventArgs e)
    {
      Command command = (Command)sender;
      if(this.UseDefaultUpdateUIMethod)
      {
        command.DefaultUpdateUI(this.IsCommand_A_Available);
      }
      else
      {
        command.Button.Enabled = this.IsCommand_A_Available;
        command.MenuItem.Enabled = this.IsCommand_A_Available;
        command.PopupMenuItem.Enabled = this.IsCommand_A_Available;
      }
    }

Both the UseDefaultUpdateUIMethod and IsCommand_A_Available properties are provided by the Form object and provide the state of the form elements. Obviously, the appropriate state properties or methods have to be provided by the application programmer in a real application.

The Command.DefaultUpdateUI method provides the same functionality as that coded in the else section. The method is presented as above only to illustrate the various properties of the Command object.

In order to create and hookup delegate's for the button and menu Click events, as well as the Command UpdateEventHandler event, the Form class is reflected and the UI elements are found from their names:

  • the ToolStripButton button is found by searching for the name {Command}Button. For the above example the actual name would be "CommandAButton",

  • the ToolStripMenuItem menu item is found from {Command}MenuItem, or "CommandAMenuItem" in the example, and

  • the ToolStripMenuItem  popup menu item is {Command}PopupMenuItem, or "CommandAPopupMenuItem" in the example.

Command provides a method to find the UI button element:


    private ToolStripButton FindButton(string buttonName)
    {
      FieldInfo buttonInfo = Target.GetType().GetField(
        buttonName,
        BindingFlags.IgnoreCase 
        | BindingFlags.Instance
        | BindingFlags.NonPublic
        | BindingFlags.Public);
      if (buttonInfo != null)
      {
        return buttonInfo.GetValue(Target) as ToolStripButton;
      }
      return null;
    }

where Target is the Form object used to construct the Command object. Similarly, the menu items can also be found.

Once the UI element objects are found, their event handler delegates can be set (or removed) by creating new delegate instances from the required method:


    private void CreateEventHandler(
      object target,
      object component,
      string eventName,
      string handlerName,
      bool add)
    {
      MethodInfo methodInfo = target.GetType().GetMethod(
        handlerName,
        BindingFlags.IgnoreCase
        | BindingFlags.Instance
        | BindingFlags.NonPublic
        | BindingFlags.Public);
      if (methodInfo != null)
      {
        EventInfo eventInfo = component.GetType().GetEvent(
          eventName,
          BindingFlags.IgnoreCase
          | BindingFlags.Instance
          | BindingFlags.Public);
        Delegate eventDelegate = Delegate.CreateDelegate(
          typeof(EventHandler), target, methodInfo);
        if (add)
        {
          eventInfo.AddEventHandler(component, eventDelegate);
        }
        else
        {
          eventInfo.RemoveEventHandler(component, eventDelegate);
        }
      }
    }

Note that the methods to be used as handlers are required to have the same signature as that required by the EventHandler:


    void MyEventHandler(object sender, EventArgs e) { ... }

Note also that the searches for variable fields, methods and events are conducted ignoring the case of the names, such that 'myEventHandler' and 'MyEventHandler' will both refer to the same construct.

Command provides a UI update method which will call the UI update handler if defined by the Form as per the requirement above:


    public event EventHandler UpdateEventHandler;
    public void Update()
    {
      if (UpdateEventHandler != null)
      {
        UpdateEventHandler(this, EventArgs.Empty);
      }
    }

Finally, all the Command's are updated as follows:


    public static void UpdateCommands()
    {
      foreach (Command command in Commands)
      {
        command.Update();
      }
    }

So,  you ask, how do I get this working in my application?

  1. First, provide a reference to the Command class by adding the C# file to your Windows.Forms application project, or reference it as a class library.

  2. Add the required UI elements to the form and name them appropriately.

  3. Add the required implementation handler and UI update handler to the code and name them appropriately.

  4. In the Form constructor after the InitializeComponent method add the following line for each Command (in the example the command is named "CommandA":

    
        Command.Commands.Add(new Command(this, "CommandA"));
  5. Add an update method to the form which is called by the Application when it is idle. This method then updates all the Command object in use:

    
        public void IdleUpdate()
        {
          Command.UpdateCommands();
        }
  6. Replace the Program class in Program.cs (.NET 2.0+), as follows:

      
      static class Program
      {
        /// <summary>
        /// The main entry point for the application.
        /// </summary>
        [STAThread]
        //static void Main()
        //{
        //    Application.EnableVisualStyles();
        //    Application.SetCompatibleTextRenderingDefault(false);
        //    Application.Run(new Form1());
        //}
        static void Main()
        {
          Application.ApplicationExit += new EventHandler(Application_ApplicationExit);
          Application.EnableVisualStyles();
          Application.SetCompatibleTextRenderingDefault(false);
          form1 = new Form1();
          Application.Idle += OnApplicationIdle;
          Application.Run(form1);
        }
        static Form1 form1;
        static EventHandler OnApplicationIdle = new EventHandler(Application_Idle);
        static void Application_Idle(object sender, EventArgs e)
        {
          form1.IdleUpdate();
        }
        static void Application_ApplicationExit(object sender, EventArgs e)
        {
          Application.Idle -= OnApplicationIdle;
        }
      }

The C# source-code for the Command class and a sample forms application is provided in a Visual Studio 2005 .NET 2.0 solution and can be downloaded from the link below:


Friday, January 19, 2007 #

spy.doc documents XML-schemas using the Altova® XMLSpy® XML-schema documentation format to produce Microsoft® HTML Help documentation (.chm) files that feature auto-generated Table-of-contents, Index, and full-text search. By providing suitably coded plugins, the documentation output can be greatly extended and enhanced.

Features

  • Windows, Console and Launcher applications.
  • Uses an XML-base project file (extension .spydoc) which can be easily edited to change application and generation settings.
  • Generates complete XMLSPY® HTML documentation for XML-schemas in an HTML Help file format.
  • Provides the same documentation settings for schema construct type, and details as provided by XMLSPY®, and also the means to order the documentation sections differently on the HTML page to the standard order provided by XMLSPY®.
  • HTML Help file is automatically generated, with complete table-of-contents, HTML content page-title based indexing, and full-text search.
  • HTML Help file is high performance, as each content page corresponds to a documentation section in the original XMLSPY® HTML documentation file.
  • Page header and footer for documentation pages can be completely user-defined.
  • The HTML page style is generated in a separate style-sheet (.css) file, allowing the styles to be easily edited in one place.
  • A separate HTML page may be defined for the HTML Help file start-page and home-page.
  • The HTML Help project file is totally user-defined with replacable parameters that are provided by the application. The HTML Help project file is input into the application as a template, from which the target .hhp file is generated which can be directly compiled.
    Automatically generates H2 and SandCastle document set name tags, which can be used for document set filtering in combined help documentation sets.
  • Provides options to retain all intermediate and temporary files, and to generate only the HTML Help build files (.hhc, .hhk, .hhp) files, so that the .chm file can easily be rebuild without having to regenerate the entire documentation set.
  • Provides for the application of user-coded plug-ins to enhance and extend the existing documentation.
  • Can generate complete documentation for large schema-sets that cannot be generated directly in one iteration by XMLSPY®.
  • No restriction on the size of the documentation, other than that imposed by the HTML Help service itself.

Function

Motivation

As the use of XML as the protocol for information interfaces is becoming more common and accepted in the industry, two important facts are emerging:

  • The W3C consortium XML-schema or XSD-standard is the major standard for describing the structure and syntax of XML data.
  • More than ever, these XML-schemas are becoming huge, complex and more resource-intensive, both in human- and computer- terms.

An important part of managing and utilizing large schema sets is to provide comprehensive and easy-to-use documentation describing the schemas. Altova® XMLSpy® provides a handy capability to do this, in both a Microsoft® Word format and an HTML format. The advantage of the HTML documentation is that it can be easily published, as well as providing the following additional important advantages:

  • The documentation provides XML-schema structure description using diagrams. The diagram constructs used have become a de-facto industry standard in terms of schema documentation. The HTML page also provides hot-spots within the diagrams that provide easy navigation between schema documentation constructs.
  • The documentation text is easy to understand and is comprehensive.
  • 'Type' and 'Used By' sections provide easy navigation around the documentation in terms of parent-child relationships.
  • Most importantly, the documentation is generated automatically, with very little user intervention.

However, when the schema(-sets) get moderately complex, the HTML documentation is inadequate, for the following reasons:

  • The documentation takes the form of a single page, the size of which can easily be in excess of a few megabytes.  This results in a slower loading page in the browser, the page navigation performance of the loaded page is poor, and the user experience is substantially degraded. For very large schemas, the page is often unable to load and even causes the browser to throw an exception.
  • The documentation has a very simple indexing section at the top of the page, which is not efficient. It has no real table-of-contents, keyword index, or full-text search, which becomes more important as the size of the documentation increases.
  • For many large schema-sets, it is impossible to generate the complete documentation, as XMLSPY® throws a memory exception and hangs up.

How it works

The spy.doc application takes as input the HTML page generated by the XMLSpy® XML-schema 'Generate Documentation' command, and outputs a fully-featured compiled HTML Help (.chm) file. spy.doc functionality includes:

  • utilizing the XMLSPY® documentation 'standard' completely;
  • automatically executes XMLSPY® via the XMLSPYLIB automation interface and generates documentation according to user-presets;
  • splits the generated single HTML documentation page into separate HTML pages;
  • builds an HTML Help table-of-contents (.hhc), and index (.hhk) file automatically from the schema construct type and construct name that it derives the contents of each page;
  • rebuilds the HTML image hot-spot and text hyperlinks automatically to retain the original XMLSPY® navigation capabilities;
  • uses a user-specified HTML Help project template (with replacable parameters), to automatically generate the target HTML Help project file (.hhp) which can directly compiled;
  • automatically executes the Microsoft® HTML Help Compiler (hhc.exe) to build the final .chm help file;
  • provides multiple XMLSPY® documentation sessions, if required, each session generating a section of the documentation. The documentation page is then split as before, then merged on a per-page basis to provide complete documentation HTML pages. These pages are then placed individually in the HTML Help file as before. Using this technique, it is possible to generate the complete documentation for large schema-sets successfully, when a single pass generation would cause XMLSPY® to fail as indicated above;
  • allows further enhancement to the generated documentation via user-coded plug-in assemblies.

The resulting HTML Help file is fully-featured, easy to navigate, and provides great performance.


Monday, November 13, 2006 #

The .NET 2.0 System.Configuration.dll provides a much more sophisticated (and complex) set of classes that access and modify application .config files, than compared to the functionality in .NET 1.x.

However, most often, the requirement is still to provide simple access to and from the key-value pairs in the appSettings section as before.

This post establishes a very simple abstract base-class that provides a simple binding to the appSettings section that requires no code to be written in the sub-class. It provides a Load method that is called in the  instance constructor. The Load method uses reflection to determine the public properties of the derived class. For each of these properties, the key-value instance in the appSettings is accessed, the value is read and converted from a string to the required property type, and the property value updated. There are two restrictions to this action:

  • The correct value type must be able to be instanced by the System.Convert.ChangeType method and converted from a System.String, so the type should be a simple (primitive) type such as a string or int, etc.
  • The property should be a simple value type which is not an array. The Load method checks for this using the System.Type.IsArray property, and ignores the property if the value is true.

Of course, it can be argued that if the application setting value does not adhere to those restrictions, it should not be stored in the appSettings section in the first place.

In the instance destructor, the Save method is called. Again using reflection, for each public property which is not an array, the key-value of the appSettings section is accessed and updated with the current value of the property. Note that the value is changed to a System.String, which is in keeping with the restrictions mentioned above. Once the settings are updated, they are saved back to the .config file and then reloaded, to ensure correct updating subsequently.

Note that the Load and Save methods are public so they can be called from external code at any time. They are also virtual so that they can be overridden by the derived class if required.

The C# code for the base-class is shown in Listing 1 below:

 

Listing 1: Application Settings Base Class
public abstract class AppSettings
{
  // Load the settings.
  public virtual void Load()
  {
    PropertyInfo[] properties = GetType().GetProperties();
    foreach (PropertyInfo property in properties)
    {
      if (!property.PropertyType.IsArray)
      {
        string appValue = ConfigurationManager.AppSettings[property.Name];
        if (appValue != null)
        {
          try
          {
            // Attempt to change the type from System.String to
            // the property type, and set the property value.
            property.SetValue(
              this,
              Convert.ChangeType(appValue, property.PropertyType),
              null);
          }
          catch {/*ignore*/}
        }
      }
    }
  }
  // Save the settings.
  public virtual void Save()
  {
    // Get the configuration file.
    System.Configuration.Configuration configuration =
      ConfigurationManager.OpenExeConfiguration(
        ConfigurationUserLevel.None);
    // Get the public properties of the class.
    PropertyInfo[] properties = GetType().GetProperties();
    // Save each property setting.
    foreach (PropertyInfo property in properties)
    {
      // Save if not an array type.
      if (!property.PropertyType.IsArray)
      {
        // Remove the setting if it exists.
        if (configuration.AppSettings.Settings[property.Name] != null)
        {
          configuration.AppSettings.Settings.Remove(property.Name);
        }
        // Add the setting.
        configuration.AppSettings.Settings.Add(
          property.Name,
          property.GetValue(this, null).ToString());
      }
    }
    // Save the configuration settings.
    configuration.Save(ConfigurationSaveMode.Modified);
    // Force a reload of the whole section.
    ConfigurationManager.RefreshSection("appSettings");
  }
  protected AppSettings()
  {
    // Load the settings when class instance is constructed.
    Load();
  }
  ~AppSettings()
  {
    // Save settings when instance is destroyed.
    Save();
  }
}

So how do you use it?

In Listing 2 is a test settings class. Note that the only requirement to get it all the work is to add the class inheritance reference. Now if your application creates an instance of the derived settings class, the .config file settings are loaded, if available. By simply destructing the instance (by normally letting it run out of scope), the settings are automatically saved for the derived class public properties. This also keeps working, whether you add or remove public properties to the derived class.

 

Listing 2: A test sub-class which is instantiable in an application.      
public class Settings : AppSettings
{
  private double doubleValue;
  public double DoubleValue
  {
    get { return doubleValue; }
    set { doubleValue = value; }
  }
  private decimal decimalValue;
  public decimal DecimalValue
  {
    get { return decimalValue; }
    set { decimalValue = value; }
  }
  private bool boolValue;
  public bool BoolValue
  {
    get { return boolValue; }
    set { boolValue = value; }
  }
  private int intValue;
  public int IntValue
  {
    get { return intValue; }
    set { intValue = value; }
  }
  private string stringValue;
  public string StringValue
  {
    get { return stringValue; }
    set { stringValue = value; }
  }
  private DateTime dateTimeValue;
  public DateTime DateTimeValue
  {
    get { return dateTimeValue; }
    set { dateTimeValue = value; }
  }
  public Settings() { }
}

Here is what the .config file looks like for this example:

 

Listing 3: The .config file showing the settings stored as keys.          
<?xml version="1.0" encoding="utf-8"?>
<configuration>
    <appSettings>
        <add key="DoubleValue" value="3.141592654" />
        <add key="DecimalValue" value="-100.01" />
        <add key="BoolValue" value="True" />
        <add key="IntValue" value="1024" />
        <add key="StringValue" value="The quick brown fox jumped over
           the lazy dog." />
        <add key="DateTimeValue" value="2006/11/13 12:34:56 AM" />
    </appSettings>
</configuration>

You can download the AppSettings class and a sample application from the link below:


Thursday, September 14, 2006 #

As an ongoing part of maintaining our CodeXS code-generation tool (read about it here: CodeXS article), we are continually looking for ways to improve the consistency, reliability and performance of the CodeXS tool.

Enter the System.Xml.Schema.XmlSchemaSet class. Unfortunately, it is available only in .NET 2.0+. Unlike the .NET 1.x XmlSchemas class, this class provides what seems to be a one-shop-stop means of reading, compiling and using a target-schema that utilizes a complex multi-namespace, multi-include/import schema set. Merely by setting up the correct URL resolver, the target-schema can simply be Add'ed to the XmlSchemaSet and all the import'ed/include'ed schemas are automatically added during the execution of that method. The XmlSchemaSet object can then be Compile'd, and the various properties of the class can be used to extract the complex types, elements etc. of the entire schema set as if it is one large XmlSchema.

One caveat though: it seems as if there are schema sets that have bi-directional schema import constructs. In these cases the same schema may be added more than once to the schema set, resulting in validation errors when the XmlSchemaSet object is Compile'd. These typically result in a 'The {object}{namespace} has already been declared.' error. An example of this schema can be found here: Global Justice XML Version 3.0.3 Schema

One way around this problem is shown in the code-excerpt below, and currently works correctly for all the schemas we are using.

Read and compile a complex XML schema set
// Set the current working directory for file-based Uri's.
FileInfo fileInfo = new FileInfo(schemaFilePath);
Directory.SetCurrentDirectory(fileInfo.DirectoryName);
XmlSchemaSet xmlSchemaSet;
// Open the target schema file as a stream.
using (StreamReader reader = new StreamReader(schemaFilePath))
{
  // XmlSchemaSet is only .NET 2.0+.
  xmlSchemaSet = new XmlSchemaSet();
  xmlSchemaSet.ValidationEventHandler += ValidationErrorHandler;
  xmlSchemaSet.XmlResolver = new XmlUrlResolver();
  XmlSchema xmlSchema = XmlSchema.Read(reader, ValidationErrorHandler);
  // Adding a target schema here will automatically resolve all
  // include/import'ed schemas - these will reside together with the
  // target schema in the XmlSchemaSet.Schemas() collection of 
  // XmlSchema objects.
  xmlSchemaSet.Add(xmlSchema);
  // Check for uniqueness - schemas may be added twice if there
  // are bi-directional imports.
  // Using an ArrayList lets use indexing for accessing the schemas.
  ArrayList schemaList = new ArrayList(xmlSchemaSet.Schemas());
  // ArrayList holding the duplicate schemas.
  ArrayList duplicateSchemaList = new ArrayList();
  for (int i = 0; i < schemaList.Count; i++)
  {
    // It seems that if the SourceUri property is unresolved 
    // (=string.Empty), then the schema has already been added.
    if (((XmlSchema)schemaList[i]).SourceUri == string.Empty)
    {
      duplicateSchemaList.Add((XmlSchema)schemaList[i]);
    }
    else
    {
      // Exhaustively test for duplicate schemas by means of the
      // SourceUri property.
      for (int j = i + 1; j < schemaList.Count; j++)
      {
        if (((XmlSchema)schemaList[i]).SourceUri ==
            ((XmlSchema)schemaList[j]).SourceUri)
        {
          duplicateSchemaList.Add((XmlSchema)schemaList[j]);
        }
      }
    }
  }
  // Remove the duplicate schemas from the set.
  foreach (XmlSchema schema in duplicateSchemaList)
  {
    xmlSchemaSet.Remove(schema);
  }
  // Compile the whole schema set.
  xmlSchemaSet.Compile();
  reader.Close();
}

The CodeXS tool is available here:

You can download the source-code by re/registering here:

You can check the latest update information here:

You can run the CodeXS online tool here:


Sunday, October 30, 2005 #

Recently I wanted to setup up an RSS feed reader and blog item display window on our website. In order to do something different (and because I'm inherently lazy), I thought it would be cool to use Microsoft's XSD.EXE tool to gen. the classes from an XML schema for RSS 2.0, write a bit of code around this and use it on our site. My primary interest was to consume the RSS feeds coming off various sites and use the blog channels I was interested in.

So, the tasks that needed doing were:

  • Validate the feeds as RSS 2.0 conformant

    I found a useful website at http://feedvalidator.org/ that allows you to submit RSS feed url's and reports on their conformance. Except for some minor issues, the feeds from today's modern blog sites are usually conformant (phew)! 

  • Find an XSD that works

    This is an issue, as there appears to be more than one 'standard' for the RSS 2.0 XML format.  After plugging many of these XSD's into XMLSpy and validating against the rss feeds, I found an XSD that worked and against which the stream can be validated. The XSD can be downloaded here: http://www.thearchitect.co.uk/schemas/rss-2_0.xsd.

  • Generate the code classes and deserialize the RSS stream

    Also an issue. XSD.EXE generates code incorrectly when a choice-item includes an enumerated type. The XmlChoiceIdentifierAttribute serialization attribute causes the reflection of the classes to fail! So what appeared to be the easy way to read RSS streams was becoming more difficult. This was resolved (and could easily be the subject of a meaty article). You can download my working code here. The cool thing about this code is that it should be very easy to generate RSS 2.0 conformant XML streams without much added code.

The working code contains both the original code-class file generated by XSD.EXE and the modified file (which works). The basis for these manual fixes can be found here:

Implementing XML Key Management Services Using ASP.NET  ('The XKMS Object Model' section)

Watch this space - I intend getting our CodeXS code generator working correctly with this schema and producing a fully-fledged RSS 2.0 serializer....(soon-ish!)


Monday, July 24, 2006 #

If you write applications in .NET that generate .NET language source-code as output, and the output language should be selectable (typically C# or VB.NET), then you will probably end up generating the output from a System.CodeDom graph. Once you have the CodeDOM the actual code generation support in .NET is easy, clean and produces reasonable code. So what's the catch? Well, consider at least the following:

  • Building CodeDOMs programmatically is a real pain - the .NET API's are convoluted, difficult to understand, error-prone and you often require many lines of code to generate one line of code in the CodeDOM. (Reminds me of the days I used to program in assembler..). For this reason, many people have gone to great pains to abstract the CodeDOM API's into more meaningful, much easier to use, and generally more efficient class libraries. I reference a few of these from CodeProject below:
  • Some of the constructs that you prefer to code yourself just cannot be reproduced in a CodeDOM, and require that you be a lot more creative and defensive in your approach when programmatically building CodeDOMs.

Another approach is to find or build a System.CodeDom.Compiler.ICodeParser implementation for the input.NET language you are generating CodeDOMs from. This allows you to write code in that .NET language, which is then parsed to produce a System.CodeDom.CodeCompileUnit (CodeDOM). Once you have the CodeDOM it is trivial to generate the code (see the CodeGeneration.GenerateCode method below). But again, there is no free-lunch - many ICodeParser implementations exist, but all of them (in my experience) are trivial solutions, mainly geared to generating code for class-interfaces or web-service interfaces.

Enter IC#Code and their #Develop (SharpDevelop) product.

As part of this product, they ship an assembly - ICSharpCode.NRefactory.dll  (and you can build the source code) which provides a fairly comprehensive ICodeParser implementation. Using the CodeDOMVisitor 'visitor' object, it is easy to generate source-code from a CodeDOM created by parsing source-code, as demonstrated in the code below. Note, though, that as before your input source-code will have to be very standard (no funny stuff), and there are some quirks with the CodeDOMVisitor object (which includes some issues with class constructor generation, and some of the looping constructs). In general the results are pleasing, and I look forward to future updates to this assembly. Interestingly, they also provide other 'visit-ations', including a working C#-to-VB.NET converter.

The code below was written for VS2005/.NET 2.0 and uses the #Develop 2.0.0.1462RC2 version source-code and DLL's. You will need to add a reference in your project to the ICSharpCode.NRefactory.dll assembly to get your project to build.

CodeGenerator Classes for .NET 2.0
using System;
using System.CodeDom;
using System.CodeDom.Compiler;
using System.Collections.Generic;
using System.Text;
using System.IO;
using ICSharpCode.NRefactory.Parser;
namespace CodeGeneration
{
  public class CodeGenerator
  {
    private ICodeParser parser;
    public ICodeParser Parser
    {
      get { return parser; }
      set { parser = value; }
    }
    private CodeDomProvider provider;
    public CodeDomProvider Provider
    {
      get { return provider; }
      set { provider = value; }
    }
    private CodeGeneratorOptions options;
    public CodeGeneratorOptions Options
    {
      get { return options; }
      set { options = value; }
    }
    protected virtual CodeGeneratorOptions 
      GetDefaultOptions()
    {
      CodeGeneratorOptions options = 
        new CodeGeneratorOptions();
      options.BracingStyle = "C";
      options.BlankLinesBetweenMembers = false;
      options.IndentString = "\t";
      return options;
    }
    public virtual void GenerateCode(TextReader reader, 
      TextWriter writer)
    {
      CodeCompileUnit codeCompileUnit = Parser.Parse(reader);
      Provider.GenerateCodeFromCompileUnit(codeCompileUnit, writer, Options);
      reader.Close();
      // Important to flush the output stream.
      writer.Flush();
      writer.Close();
    }
    public CodeGenerator(ICodeParser parser, 
      CodeDomProvider provider)
    {
      Options = GetDefaultOptions();
      Parser = parser;
      Provider = provider;
    }
    public CodeGenerator(ICodeParser parser, 
      CodeDomProvider provider,
      CodeGeneratorOptions options)
    {
      Options = options;
      Parser = parser;
      Provider = provider;
    }
  }
  public class Parser : ICodeParser
  {
    #region ICodeParser Members
    public CodeCompileUnit Parse(TextReader codeStream)
    {
      // Set the parser for the correct language.
      IParser parser = ParserFactory.CreateParser(Language,
        codeStream);
      parser.Parse();
      // Use a CodeDom visitor to gen. a CodeCompileUnit.
      CodeDOMVisitor visit = new CodeDOMVisitor();
      CodeNamespace globalNamespace =  (CodeNamespace)
        visit.Visit(parser.CompilationUnit, null);
      // Mostly, you need to remove the "Global"
      // code namespace from the compile unit.
      visit.codeCompileUnit.Namespaces.Remove(globalNamespace);
      return visit.codeCompileUnit;
    }
    #endregion
    private SupportedLanguage language = SupportedLanguage.CSharp;
    public SupportedLanguage Language
    {
      get { return language; }
      set { language = value; }
    }
    public Parser(SupportedLanguage language)
    {
      Language = language;
    }
  }
}

Thursday, July 13, 2006 #

CodeXS Version 0.57b was released on 13 July 2006.

You can download the source-code by re/registering here:

You can check the latest update information here:

You can run the CodeXS online tool here:

Version 0.57ß - 13 Jul 2006

  • Currently still provides complete support for VS2005/.NET 2.0, and VS2003/.NET 1.1.
  • Collection class bug-fixes

The following bugs-fixes are implemented:

  • Names for schema complex types an elements with multiple occurrences can now include the phrase 'Collection'.
  • Return types and parameters for the standard collection methods and properties now use the correct collection element type.
  • The ToArray and FromArray corrected for double-index ragged array return and parameter types, respectively.

Note: CodeXS supports ragged array serialization with a maximum array rank of 2.

  • Override modifier on sub-classed virtual methods and properties

A more general solution has been provided to correct the error where a virtual attribute is declared that should be an override attribute.

  • Type instantiation support

Type instantiation (vs. primitive non-instantable types) checking is enhanced. This should reduce the occurrence of the (usually recoverable) exception: "Cannot determine type <type>''. Type assumed instantiable.".

  • Additional documentation support

Documentation utility methods have been added to allow for more extensive and easier documentation of the generated code.

  • ISchemaObjectReader interface removed

This interface has been removed for the following reasons:

  • The ICodeModifer interface provides the correct entry points for code modifer enhancements requiring direct reading from the XML schemas.
  • The ICodeModifer interface now supports the .NET 2.0 SchemaImporterExtension class implementation in the ICodeModifier interface.
  • The ICodeModifer interface supports initialization, which can be used as an entry point to pre-read schemas.
  • Logging support for ICodeModifier implementers

Logging using the CodeXS logging implementation can be used directly from an ICodeModifier implementor in an external assembly. 

  • Missing declarations and definitions in the code

A bug-fix was implemented to avoid this error, and appears to be working as expected.

  • Assembly and file version control

Assembly and file version numbers now correspond with the public release version numbers.

  • XML schema documentation add-in

An ICodeModifier implementation add-in has been provided which reads schema <xs:documentation/> tags and injects the documentation into the generated code documentation comments.

This add-in is implemented by reading the XML schema(s) for the generated code files, and matching schema-types to generated code types. This add-in is implemented to read and modify the CodeDOM's after the other code-generators have executed and therefore post-processes the CodeDOM. It has been implemented to work for both .NET 2.0, and .NET 1.1.

For .NET 2.0, the CXSC application commandline is:

cxsc {..} -m="CodeXS.Schema.StandardCodeModifier.dll","SchemaDocumentation.dll"

For .NET 1.1, the CXSC application commadline is:

cxsc {..} -m="CodeXS.Schema.StandardCodeModifier.Net1_1.dll","SchemaDocumentation.Net1_1.dll"

NOTE: The CodeXS online tool does not utilize this add-in.

  • .NET 2.0 SchemaImporterExtension class

For .NET 2.0, the abstract class System.Xml.Serialization.Advanced.SchemaImporterExtension has been provided for sub-classing. A sub-classed instance receives call-backs via the sub-class implemented ImportSchemaType methods, and provides a context for the CodeDOM type/member and the System.Xml.Schema.XmlSchemaObject that is used for the CodeDOM construct. This allows a code-modifier to modify the CodeDOM while it is being constructed by the code generator. In general, this provides a much safer and simpler way to relate the XML schema objects to their CodeDOM counterparts.

CodeXS provides complete support for writing SchemaImporterExtension implementers.


Wednesday, June 21, 2006 #

CodeXS Version 0.56b was released on 19 June 2006.

You can download the source-code by re/registering here:

You can check the latest update information here:

You can run the CodeXS online tool here:

Changes in Version 0.56ß - 19 Jun 2006

  • Currently still provides complete support for VS2005/.NET 2.0, and VS2003/.NET 1.1.
  • Updated collection object generated code

Updated the collection object generated code:

  • Re-implemented the FromArray method with less error prone code.
  • Re-implemented the ToArray method with less error prone code.
  • Corrected the return value for the Add method. Now returns int instead of void.

My thanks to Dave Waterworth for this fix - Willem Fourie

  • New field names

This version uses a different scheme when renaming class field members. This resulted from various XML schemas using elements with the name 'id' and 'Id', respectively. In prior versions this resulted in the same field name '__Id'. In this version they are distinct: '__id' and '__Id', respectively.

  • Export code exception fix

During the import/export of the XmlTypeMapping objects, the import/export sequence is now done via an explicit array list object. This avoids the exception: 'collection changed during iteration'.

  • Handling of XSD elements and XSD types with the same name

In prior versions, this caused a code generation exception. However, often XML schemas are defining elements with the same name as their type names. For those cases, the single class generated is correct, as it represents completely both the element and the type. For this reason, the prior exception has been changed to provide a warning message in the CodeXS.log file, and generation continues.

It is strongly recommended that these warning messages are noted, as they do provide notification of potential de-/-serialization problems when the generated code is used.

  • XmlChoiceIdentifierAttribute handling

This attribute was not handled correctly in prior versions, resulting in de-/-serialization exceptions. This is now working and relies on the following changes in this version:

  • The class properties intended for de-/-serialization now set/get the original field types: In prior versions, if the property set/get type was an ObjectCollection, it now sets/gets an object[] array object, which is the original type generated in the CodeDOM. The underlying field is still an ObjectCollection type, and the collection ToArray and FromArray methods are used to get/set (respectively) the field correctly. Using the original scalar types for de-/-serialization avoids the exception: 'ItemsElementName should be of type []'.
  • To support XML schemas that include an xs:any element as part of the enumeration choice types, all XmlAnyElementAttribute attributes for a member that also has an XmlChoiceIndentiferAttribute attribute are removed.
  • To support XML schemas that include an xs:any element as part of the enumeration choice types, all System.Xml.Serialization.XmlEnumAttribute("##any:") attributes are removed from the choice identifier enumerated type members.
  • The attribute name is changed from 'ItemsElementName' to '_ItemsElementName', and the normal property generation correctly generates the _ItemsElementName array set/get property, using the FromArray and ToArray methods of the referenced collection class.

Monday, June 19, 2006 #

A while ago I posted this post on my blog:

Since that post, our CodeXS code-generator tool has been updated. With the latest update of the tool due out this week, the code-generator correctly generates the de-/-serialization code to read and write RSS 2.0 feeds. It is now fairly simple to also create RSS feeds, as feed item objects can be easily created and added to the items collection, instead of trying to manipulate arrays.

A simple example that uses the updated code can be downloaded here:

The code language is C# and the solution and project files are for VS2003/.NET 1.1, but should migrate easily to VS2005/.NET 2.0.


Wednesday, May 31, 2006 #

Following my prior post:

I have had many requests to provide the same code in unmanaged C++ - so here goes:

 

// KeyDecoder.h
#pragma once
#ifndef byte
typedef unsigned char byte;
#endif // byte
class KeyDecoder
{
public:
  enum Key { XP, Office10, Office11 };
  static int DecodeProductKey(KeyDecoder::Key key, char* pDecodedKey);
protected:
  static byte* GetRegistryDigitalProductId(KeyDecoder::Key key);
  static char* DecodeProductKey(byte* digitalProductId);
};

and:

 

// KeyDecoder.cpp
#include "StdAfx.h"
#include "keydecoder.h"
int KeyDecoder::DecodeProductKey(KeyDecoder::Key key, char* pKey)
{
  byte* pEncodedPID = GetRegistryDigitalProductId(key);
  int keyLen = 0;
  if(pEncodedPID)
  {
    char* pDecodedPID = DecodeProductKey(pEncodedPID);
    if(pDecodedPID)
    {
      keyLen = (int)::strlen(pDecodedPID);
      if(pKey)
      {
        ::strcpy(pKey, pDecodedPID);
      }
      delete[] pDecodedPID;
    }
  }
  return keyLen + 1;
}
char* KeyDecoder::DecodeProductKey(byte* digitalProductId)
{
  // Offset of first byte of encoded product key in 
  //  'DigitalProductIdxxx" REG_BINARY value. Offset = 34H.
  const int keyStartIndex = 52;
  // Offset of last byte of encoded product key in 
  //  'DigitalProductIdxxx" REG_BINARY value. Offset = 43H.
  const int keyEndIndex = keyStartIndex + 15;
  // Possible alpha-numeric characters in product key.
  char digits[] = 
  {
    'B', 'C', 'D', 'F', 'G', 'H', 'J', 'K', 'M', 'P', 'Q', 'R', 
    'T', 'V', 'W', 'X', 'Y', '2', '3', '4', '6', '7', '8', '9',
  };
  // Length of decoded product key
  const int decodeLength = 29;
  // Length of decoded product key in byte-form.
  // Each byte represents 2 chars.
  const int decodeStringLength = 15;
  // Array of containing the decoded product key.
  char* pDecodedChars = new char[decodeLength + 1];
  ::memset(pDecodedChars, 0, decodeLength + 1); 
  // Extract byte 52 to 67 inclusive.
  byte hexPid[keyEndIndex - keyStartIndex + 1];
  for (int i = keyStartIndex; i <= keyEndIndex; i++)
  {
    hexPid[i - keyStartIndex] = digitalProductId[i];
  }
  for (int i = decodeLength - 1; i >= 0; i--)
  {
    // Every sixth char is a separator.
    if ((i + 1) % 6 == 0)
    {
      *(pDecodedChars + i) = '-';
    }
    else
    {
      // Do the actual decoding.
      int digitMapIndex = 0;
      for (int j = decodeStringLength - 1; j >= 0; j--)
      {
        int byteValue = (digitMapIndex << 8) | hexPid[j];
        hexPid[j] = (byte)(byteValue / 24);
        digitMapIndex = byteValue % 24;
        *(pDecodedChars + i) = digits[digitMapIndex];
      }
    }
  }
  return pDecodedChars;
}
byte* KeyDecoder::GetRegistryDigitalProductId(KeyDecoder::Key key)
{
  HKEY hKey = 0;
  LONG lResult = 0L;
  const char* pPIDName = "DigitalProductId";
  byte* pPID = 0;
  switch(key)
  {
    case KeyDecoder::Key::XP:
      lResult = ::RegOpenKeyEx(HKEY_LOCAL_MACHINE,
        "SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion",
        0, KEY_QUERY_VALUE, &hKey);
      break;
    case KeyDecoder::Key::Office10:
      break;
    case KeyDecoder::Key::Office11:
      break;
  }
  if(lResult == ERROR_SUCCESS)
  {
    DWORD cbData = 0;
    DWORD dwType = 0;
    lResult = ::RegQueryValueEx(hKey, pPIDName, 0, &dwType, 0, &cbData);
    if(lResult == ERROR_SUCCESS)
    {
      pPID = new byte[cbData];
      lResult = ::RegQueryValueEx(hKey, pPIDName, 0, &dwType, pPID, &cbData);
      if(lResult != ERROR_SUCCESS)
      {
        delete[] pPID;
        pPID = 0;
      }
    }
  }
  if(hKey)
  {
    ::RegCloseKey(hKey);
  }
  return pPID;
}

 

This provides the same functionality as the C# code in the prior post.

XPKey is a standard Win32 console application that pops up a message box containing the decoded XP product key. It also copies the product key to the clipboard. You can download the executable here:

and the VC source-code and project file here:

The C++ application is provided for VS2003, but should convert without problems to VS2005. It uses only standard Win32 calls - no MFC/ATL/STL etc.

PLEASE NOTE: All code listed here is provided as-is, with no guarantees what so ever. Use of this code does not require a license and no copyright on the code exists or is implied. You are free to use the code as you see fit, for any commercial or non-commercial use.


Tuesday, May 30, 2006 #

Maybe this is the external tool that MS forgot to build into VS...?

I often find that I need to open a File Explorer window in the folder that contains the file highlighted in the VS Solution Explorer. Manual navigation can be frustrating, and time-consuming.

The easy solution is to define an external tool in the VS IDE as follows:

  • Click the Tools/External Tools... menu item.
  • In the External Tools Dialog, click the Add button.
  • In the Title text box add: Explorer,
  • In the Command text box add: explorer.exe,
  • In the Arguments text box add: /select,"$(ItemPath)",
  • In the Initial Directory text box add: $(ItemDir).
  • Click Ok.

Now when you have a file highlighted in the solution window, click the Tools/Explorer menu item in the VS IDE, and File Explorer opens up into the appropriate folder with the highlighted file in the VS Solution window selected in File Explorer.

Works for VS2003, and VS2005.


Wednesday, May 17, 2006 #

x.doc is a Visual Studio (VS) 2005 add-in that provides a means to manage and visualize source-code comment documentation interactively in the IDE. A screen-shot is shown below: 

Features

x.doc provides two windows:

  • The Visualizer window which is the window that provides the rendered XML documentation text using a defined set of XSL templates and a CSS. This window is interactively updated as the caret is moved in a code-editor window. If the documentation is not well-formed XML or an error occurs in the rendering, an error is displayed in this window. If the caret is not positioned in a documentation block, the message "No documentation" is displayed. This window is implemented to work as a standard 'Tool Window' in the IDE, and can be floated, docked etc. like any other tool window.
  • The Output window which is similar to the IDE build window in that it displays the error with the appropriate source-code file path and line and character position offsets. Double-clicking the error line in this window will move the caret to the referenced editor window and offset in the code at which the error has occurred. It shares the standard output window and its content is displayed by selecting the x.doc menu-item in the 'Show output from:' selector.

x.doc provides a menu tool-strip above the Visualizer window, and is described below as they are ordered on the tool-strip (from left-to-right):

  • Prior Documentation Item: This button command moves the caret to the prior documentation block in the code editor window and selects the block as shown in the diagram above.
  • Next Documentation Item: This command moves the caret to the next documentation block.
  • Collapse All Documentation: This command traverses the entire code document and outlines the documentation blocks. Each documentation block is shown only by three ellipses ('...') and is an effective way to quickly hide all the documentation.
  • Import Documentation Item: This button command imports the documentation from an external file and embeds it as a documentation block in the source-code. This will only happen if the caret is positioned over a single-line documentation block containing a well-formed tag that points to a valid external documentation file with a name attribute that can be resolved to a documentation item in the external file.
  • Export Documentation Item: This command reverses the action of the 'Import Documentation Item' above. The documentation block is exported to the external documentation file and a suitably formed tag is substituted in its place. (See The Code section for constraints on the use of this tag). Note that badly-formed XML is exported "as-is", so it is best to check the Output window for errors before exporting the documentation block.
  • Import All Documentation: This command works the same as the "Import Documentation Item" above, except that it traverses the entire source-code file and imports all documentation.
  • Export All Documentation: This command exports all documentation to the external documentation file, reversing the action executed by 'Import All Documentation' above.
  • Documentation Display Style Selector: This selection list will accommodate any number of XSL/CSS templates which are imported from external .xsl and .css files when the add-in is first loaded. The XSL/CSS files x.doc.xsl and x.doc.css are included with the x.doc add-in installer and provides a pseudo-MSDN documentation look-and-feel. Obviously, these files can be modified or more style files added, which can then be individually selected, causing the documentation to be immediately re-rendered using the selected style. It is also possible to extend the VS IDE documentation tags and provide customized documentation functionality. NDoc already does this to provide the various standards of documentation-styles, including their MSDN style, which is exactly like the Microsoft H1 and H2 Help.

x.doc is completely NDoc compatible, in that the XML files that are generated by VS when you compile with the /doc:file option can be used as-is with NDoc. This applies whether the documentation is read directly from the source-code, or from an external file.


Sunday, April 23, 2006 #

Ever needed to reinstall your PC and could not recover your product key from that 'safe storage' you were supposed to use? It is no secret that if your system is still running you can recover the key from the system registry where it is encoded in a REG_BINARY value.

You can download a free utility that recovers the XP product key (and the MS Office product key if its installed):

You can even recover it online here:

but if you're paranoid, maybe these applications are actually stealing your product key, so you want to do this yourself with your own code, right?  The only bit of code I could find that can recover product keys is here:

which provides a working Delphi source-code listing. I only required to find the XP product key, so after hacking the code, I produced a working C# version (works with .NET 1.1 and 2.0) shown below. I allowed place-holders for other product keys but the code may have to be further modified to work correctly.

 

Windows XP KeyFinder Listing
using System;
using System.Collections;
using Microsoft.Win32;

namespace MSKeyFinder
{
  public class KeyDecoder
  {
    public enum Key { XP, Office10, Office11 };
    public static byte[] GetRegistryDigitalProductId(Key key)
    {
      byte[] digitalProductId = null;
      RegistryKey registry = null;
      switch(key)
      {
        // Open the XP subkey readonly.
        case Key.XP:
          registry = 
            Registry.LocalMachine.
              OpenSubKey(
                @"SOFTWARE\Microsoft\Windows NT\CurrentVersion",
                  false);
          break;
        // Open the Office 10 subkey readonly.
        case Key.Office10:
          // TODO: Open the registry key.
          break;
        // Open the Office 11 subkey readonly.
        case Key.Office11:
          // TODO: Open the registry key.
          break;
      }
      if(registry != null)
      {
        // TODO: For other products, key name maybe different.
        digitalProductId = registry.GetValue("DigitalProductId")
          as byte[];
        registry.Close();
      }
      return digitalProductId;
    }
    public static string DecodeProductKey(byte[] digitalProductId)
    {
      // Offset of first byte of encoded product key in 
      //  'DigitalProductIdxxx" REG_BINARY value. Offset = 34H.
      const int keyStartIndex = 52;
      // Offset of last byte of encoded product key in 
      //  'DigitalProductIdxxx" REG_BINARY value. Offset = 43H.
      const int keyEndIndex = keyStartIndex + 15;
      // Possible alpha-numeric characters in product key.
      char[] digits = new char[]
      {
        'B', 'C', 'D', 'F', 'G', 'H', 'J', 'K', 'M', 'P', 'Q', 'R', 
        'T', 'V', 'W', 'X', 'Y', '2', '3', '4', '6', '7', '8', '9',
      };
      // Length of decoded product key
      const int decodeLength = 29;
      // Length of decoded product key in byte-form.
      // Each byte represents 2 chars.
      const int decodeStringLength = 15;
      // Array of containing the decoded product key.
      char[] decodedChars = new char[decodeLength];
      // Extract byte 52 to 67 inclusive.
      ArrayList hexPid = new ArrayList();
      for (int i = keyStartIndex; i <= keyEndIndex; i++)
      {
        hexPid.Add(digitalProductId[i]);
      }
      for (int i = decodeLength - 1; i >= 0; i--)
      {
        // Every sixth char is a separator.
        if ((i + 1) % 6 == 0)
        {
          decodedChars[i] = '-';
        }
        else
        {
          // Do the actual decoding.
          int digitMapIndex = 0;
          for (int j = decodeStringLength - 1; j >= 0; j--)
          {
            int byteValue = (digitMapIndex << 8) | (byte)hexPid[j];
            hexPid[j] = (byte)(byteValue / 24);
            digitMapIndex = byteValue % 24;
            decodedChars[i] = digits[digitMapIndex];
          }
        }
      }
      return new string(decodedChars);
    }
  }
}

Once you realize how the product keys are encoded, a search through the registry for the values starting with DigitalProductId indicates that many more product keys may be encoded this way.

PLEASE NOTE: All code listed here is provided as-is, with no guarantees what so ever. Use of this code does not require a license and no copyright on the code exists or is implied. You are free to use the code as you see fit, for any commercial or non-commercial use.


Friday, April 07, 2006 #

The CodeXSWebservice is the same webservice used by our CodeXS Online Tool to generate .NET C# and VB code directly from XML schemas (XSD'es)

The CodeXSWebservice has been made available on an experimental basis. Point your web-reference to:

http://www.bware.biz/CodeXS/CodeXS.asmx

Currently, the input schema has to be located at a publicly available URL, but we will probably extend the functionality to allow multiple XSD files to be input as a ZIP-archive, and to return the generated code in a ZIP-archive. This functionality is already supported by the Online Tool.