General C# Programming

Create Strong Named Assemblies - The Easy Way

Alois Kraus — Tue, 23 Jan 2007 20:21:00 GMT

Strong naming has always been a complicated matter. Sometimes it is even impossible when you get unsigned assemblies from a third party software vendor and you have to build strong named assemblies to install them into the GAC. When you decide to strong name your project it is a all or nothing decision. If your target is strong named then all references must also be strong named. If you do not have the source code and a project for your third party dll you will not be able to sign your project. I have created Signer which is hosted at CodePlex that allows you to add strong names to binaries within minutes. You can even create a strong named build of your whole project without any changes to your build settings. Below is a picture which shows Signer in action:

How does it work?

Signer does basically a full round trip by decompiling the assembly into IL code make the necessary modifications and compile it back to a valid assembly. The required modifications include

Update of all references
Change/Removal of InternalsVisibleToAttribute
Update of custom attributes with a type parameter
A little fix to work around an ILDASM problem

Reference Update in IL

This is the easy part. A reference does consist of the assembly name, public key token and it's version. Signer has only to insert the public key token where none is present to patch the IL file.

.assembly extern Microsoft.Practices.EnterpriseLibrary.Common
{
.publickeytoken = (BE CA 05 5E 5B 7D 2D C8 ) // Inserted by Signer if not present
.ver 2:0:0:0
}

InternalsVisibleTo Attribute

With .NET 2.0 the InternalsVisibleTo attribute was introduced that allows to grant other assemblies access to classes and methods marked with the internal keyword. To prevent in a strong name scenario that an unsigned assembly does get access to internals directly (you can use reflection instead but this comes with a perf cost) the CLR does check at runtime if all InternalsVisibleTo attribute declarations grant access to strong named assemblies only. If not a runtime exception is thrown and the process terminates.

.custom instance void [mscorlib]System.Runtime.CompilerServices.InternalsVisibleToAttribute::.ctor(string) =
   ( 01 00 48 4D 69 63 72 6F 73 6F 66 74 2E 50 72 61   // ..HMicrosoft.Pra
                                63 74 69 63 65 73 2E 45 6E 74 65 72 70 72 69 73   // ctices.Enterpris
                                65 4C 69 62 72 61 72 79 2E 4C 6F 67 67 69 6E 67   // eLibrary.Logging
                                2E 43 6F 6E 66 69 67 75 72 61 74 69 6F 6E 2E 44   // .Configuration.D
                            65 73 69 67 6E 2E 54 65 73 74 73 00 00 )          // esign.Tests.."

You have to insert the public key which is over 300 bytes long into your project which is quite tedious if you want to do it manually. At the moment Signer makes life easy and simply removes the attribute to prevent run time errors but if needed the change is trivial.

Custom Attributes
This was by far the most tricky part. When you declare custom attributes with a type parameter the full qualified type name including assembly and public key token is inserted into the IL code. Normally you get this as a binary blob but with the /CAVERBAL option of ILDASM which was introduced with .NET 2.0 things get much easier.

.custom instance void [System]System.ComponentModel.EditorAttribute::.ctor(class [mscorlib]System.Type,
                                                                           class [mscorlib]System.Type)
= {
type(class
'Microsoft.Practices.EnterpriseLibrary.Configuration.Design.ReferenceEditor,
Microsoft.Practices.EnterpriseLibrary.Configuration.Design, Version=2.0.0.0, Culture=neutral,
    PublicKeyToken=beca055e5b7d2dc8')
type(class
  'System.Drawing.Design.UITypeEditor, System.Drawing, Version=2.0.0.0, Culture=neutral,
   PublicKeyToken=b03f5f7f11d50a3a')
}

Signer does parse the attributes and does set the PublicKeyToken where needed.

ILDASM Problems
All would work wonderful if, well if the nice /CAVERBAL option would produce running code. But I got many ConfiguratonErrors exceptions which told me that the default value was of the wrong type. Arghhh. After digging a little deeper I found out what was going on. Suppose you have a simple configuration class for settings inside the App.Config file:

using System;

using System.Collections.Generic;

using System.Text;

using System.Configuration;

namespace NoRoundTrip

{

enum MyValues

{

}

class Check : ConfigurationSection

{

const string PropName = "Name";

// DefaultValue is a property of type object

[ConfigurationProperty(PropName,DefaultValue=MyValues.V1)]

public MyValues Value

{

get { return (MyValues) this[PropName]; }

}

class Program

{

static void Main(string[] args)

{

Check c = new Check();

MyValues defValue = c.Value; // get default value

}

The program will work without problem when used as is. But after decompiling with the /CAVERBAL option and recompiling you will get a nasty runtime exception:

Unhandled Exception: System.Configuration.ConfigurationErrorsException:
The default value for the property 'Name' has different type than the one of the property itself.

Here is the innocent looking IL code
    .custom instance void
[System.Configuration]System.Configuration.ConfigurationPropertyAttribute::.ctor(string)
        = {
string('Name')
            property object 'DefaultValue' = object( int32(0) )
  }

The ConfigurationProperty DefaultValue does not preserve type identity for enums but does simply declare them as its native integer value which will lead to this exception during run time because there is an extra check code in the BCL code. The generated IL code is valid for the CLR but not for the BCL. Signer does get around this issue by decompiling every assembly once with the parser friendly /CAVERBAL option and the second time without. It will then merge the binary blobs for the ConfigurationProperty Attributes into the other IL file to get a fully functional assembly. Once this obstacle was solved all worked perfectly.

Signer Usage

If you start Signer without any options you will get a pretty self explanatory help to do signing. The only thing to remeber is that the .NET Framework tool sn is needed to generate a strong name key pair by simply executing sn -k c:\key.snk and you are ready to use Signer out of the box with your project.

Signer (C) by Alois Kraus 2007 (akraus1@gmx.de) version 1.0.0.0

Signer -k <KeyFile> -outdir <Output Dir> -a <Assembly1> <Assembly2> ... [-decomp "Options" -comp "Options" -debug]
   Signer does sign all assemblies and updates their references if the assembly is not already signed
   This way you can convert a whole bunch of assemblies which have never seen a key file into strong named assemblies without recompiling!
   -k        Key File name of key container file which is created usually by the sign tool (sn -k)
   -outdir   Output directory where the strong signed assemblies are copied to
   -decomp   Additional ildasm options e.g. "/STATS"
   -comp     Additional ilasm options e.g. "/X64"
   -a        Can be a list of assembly names or a wildcard search pattern e.g. *.dll
   -debug    Do not delete temporary IL files in %TEMP% flder after completion

Example: Signer -k c:\key.snk -outdir .\build -a *.dll *.exe Copy the signed assemblies into the newly created directory build.
Example: Signer -k c:\key.snk -outdir . -a *.dll *.exe        Sign and overwrite the orignal binaries in current directory.

Error: No Key file specifed (use -k KeyFile ).
Error: No Output directory specifed (use -outdir Dir ).
Error: No input assemblies specified (use -a Assembly ).

Conclusions

It was fun to create the tool while learning quite a lot about the assembly structure, strong names and IL. As you can see strong names are not really a protection for your software as they can be easily removed and your assemblies resigned without big problems. The only real advantage strong names offer is that you can prove that an assembly which has your public key on it was created by you as long as your secret key is kept safe.
From this point other extensions to Signer are quite easy to add. E.g. removal of strong names, make internals public or adding your own InternalsVisibleTo attribute to other assemblies. The world is full of possibilities ;-).

Cooperative Application Shutdown with the CLR

Alois Kraus — Mon, 30 Oct 2006 06:12:00 GMT

All good things have to end even your perfectly working managed executable. But do you know in what circumstances the CLR will terminate your program and much more importantly when do you have a chance to run some finalizers and shutdown code? As always in live it depends. Lets have a look at the reasons why program execution can be terminated.

Ways to terminate an application:

The last foreground thread of an application ends. The thread which entered the Main function is usually the only foreground thread in your application.
If you run an Windows Forms application you can call System.Windows.Forms.Application.Exit() to cause Application.Run to return.
When you call System.Environment.Exit(nn).
Pressing Ctrl-C or Ctrl-Break inside a Console application.
Call System.Environment.FailFast (new in .NET 2.0) to bail out in case of fatal execution errors.
An unhanded exception in any thread regardless if it is managed or unmanaged.
Unmanaged exit calls within unmanaged code.

What exactly happens during shutdown is explained by an excellent article of Chris Brumme.

Shutdown Process
Generally speaking we can distinguish between two shutdown types: Cooperative and Abnormal. Cooperative means that you get some help from the CLR (execution guarantees, callback handlers, ...) while the other form of exit is very rude and you will get no help from the runtime.
During a cooperative shutdown the CLR will unload the Application Domain (e.g. after leaving the Main function). This involves killing all threads by doing a hard kill as opposed to the "normal" ThreadAbortException way where the finally blocks of each thread are executed. In reality the threads to be killed are suspended and never resumed. After all threads sleep the pending finalizers are executed inside the Finalizer thread. Now comes a new type of finalizers into the game which was introduced with .NET 2.0. The Critical Finalizers are guaranteed to be called even when normal finalization did timeout (for .NET 2.0 the time is 40s ) and further finalizer processing is aborted. If you trigger an exception inside a finalizer and it is not catched you will stop any further finalization processing, including the new Critical Finalizers. This behavior should be changed in a future version of the CLR. What are these "safe" finalizers good for when an "unsafe" finalizer can prevent them from running? There is already a separate critical finalizer queue inside the CLR which is processed after the normal finalization process did take place.

Normal Shutdown
A fully cooperative shutdown is performed in case of 1, 2 and 3. All managed threads are terminated without further notice but all finalizers are executed. No further notice means that no finally blocks are executed while terminating the thread.

Abnormal Shutdown
The bets are off and no CLR event is fired which could trigger a cleanup action such as calling finalizer, or some events. Case 4 can be converted into a normal shutdown by the code shown below. Case 6 can be partially handled inside the AppDomain.UnhandledException. The other ones (Environment.FailFast and unmanaged exits) have no knowledge of the CLR and perform therefore a rude application exit. The only exception is the Visual C++ (version 7+) Runtime which is aware of the managed world and calls back into the CLR (CorExitProcess) when you call the unmanaged exit() or abort() functions.

Interesting Events
All these events are only processed when we are in a cooperative shutdown scenario. Killing your process via the Task Manager will cause a rude abort where none of our managed shutdown code is executed.

The AppDomain.DomainUnload event is not called in your default AppDomain. When you exit e.g. the Main function the default AppDomain is unloaded but your event handler will never be called. The DomainUnload handler is called from an arbitrary thread but always in the context of the domain that is about to be unloaded.
AppDomain.ProcessExit is called when the CLR does detect that a shutdown has to be performed. You can register in each AppDomain such a handler but be reminded that after the DomainUnload event was fired you will never see a ProcessExit event because your AppDomain is already gone. These two events are mutually exclusive. When you kill the process via
AppDomain.UnhandledException is called only in the default AppDomain. You can register this handler only in the "root" AppDomain which is the first one created for you by the CLR. Hosting of other AppDomains in a plugin architecture can cause unexpected side effects if the loaded modules register this handler and expect it to be called. When the handler is left the process finally terminates which cannot be prevented in .NET 2.0 except if you set a compatibility flag inside the runtime settings your App.config file.
```
 <legacyUnhandledExceptionPolicy enabled="1"/>
```

Ctrl-C/Ctrl-Break
The default behavior of the CLR is to do nothing. This does mean that the CLR is notified very late by a DLL_PROCESS_DETACH notification in which context no managed code can run anymore because the OS loader lock is already taken. We are left in the unfortunate situation that we get no notification events nor are any finalizers run. All threads are silently killed without a chance to execute their catch/finally blocks to do an orderly shutdown. I said in the first sentence default because there is a way to handle this situation gracefully. The Console class has got a new event member with .NET 2.0: Console.CancelKeyPress. it allows you to get notified of Ctrl-C and Ctrl-Break keys where you can stop the shutdown (only for Ctrl-C but not Ctrl-Break). If you want to do a coordinated shutdown inside the handler you will run into the gotcha that calling Environment.Exit does not trigger any finalizers. When we do have other plans to exit the process we need to spin up a little helper thread inside the event handler which will then call Environment.Exit. Voila our finalizers are called.

Graceful Shutdown when Ctrl-C or Ctrl-Break was pressed.

static void GraceFullCtrlC()

{

Console.CancelKeyPress += delegate(object sender, ConsoleCancelEventArgs e)

{

if( e.SpecialKey == ConsoleSpecialKey.ControlBreak )

{

Console.WriteLine("Ctrl-Break catched and translated into an cooperative shutdown");

// Envirionment.Exit(1) would NOT do a cooperative shutdown. No finalizers are called!

Thread t = new Thread(delegate()

{

Console.WriteLine("Asynchronous shutdown started");

Environment.Exit(1);

});

t.Start();

t.Join();

}

if( e.SpecialKey == ConsoleSpecialKey.ControlC )

{

e.Cancel = true; // tell the CLR to keep running

Console.WriteLine("Ctrl-C catched and translated into cooperative shutdown");

// If we want to call exit triggered from out event handler we have to spin

// up another thread. If somebody of the CLR team reads this. Please fix!

new Thread(delegate()

{

Console.WriteLine("Asynchronous shutdown started");

Environment.Exit(2);

}).Start();

}

};

Console.WriteLine("Press now Ctrl-C or Ctrl-Break");

Thread.Sleep(10000);

}

Unhandled Exceptions

It is very easy to screw up your application with an abnormal application exit. Lets suppose the following code:

static void RudeThreadExitExample()

{

// Create a new thread

new Thread(delegate()

{

Console.WriteLine("New Thread started");

// throw an unhandled exception in this thread which will cause the application to terminate

// without calling finalizers and any other code.

throw new Exception("Uups this thread is going to die now");

}).Start();

}

We spin up an additional thread and create an unhandled exception. It is possible to register the AppDomain.Unhandled exception handler where you can e.g. log the exception but nonetheless your application will be rudely terminated. No finalizers are executed when an unhandled exception occurs. If you try to fix this by calling Environment.Exit inside your handler nothing will happen except that the ProcessExit event is triggered. We can employ our little Ctrl-C trick here and force an ordered cooperative shutdown from another thread.

Cooperative Unhandled Exception Handler (Finalizers are called)

static void CurrentDomain_UnhandledException(object sender, UnhandledExceptionEventArgs e)

{

Console.WriteLine("Unhandled exception handler fired. Object: " + e.ExceptionObject);

// Prepare cooperative async shutdown from another thread

Thread t = new Thread(delegate()

{

Console.WriteLine("Asynchronous shutdown started");

Environment.Exit(1);

});

t.Start();

t.Join(); // wait until we have exited

}

This way you can ensure that your application exits in a much cleaner way where all finalizers are called. What I found interesting during my investigation that there seems no time limit to exist how long I want to run the unhandled exception handler. It is therefore possible that you have several unhandled exception handlers running at the same time. An ordered application shutdown can be quite twisted if you want to do it right. Armed with the knowledge from this post you should have a much better feeling what will happen when things go wrong now.

Lazy Vs Eager Init Singletons / Double-Check Lock Pattern

Alois Kraus — Mon, 11 Sep 2006 01:36:00 GMT

Singletons are the easiest programming pattern to learn which seems to be the reason why it is often used in places where it is inappropriate. They are very straightforward to implement and understand. Just create a static field and return the static instance in a public get property. A Logger Singleton is a classic example of this pattern:

public class Logger
{

static LogWriter writer = new LogWriter();

public static LogWriter Writer

{

get { return writer; }

}

If you wrote such code before and thought that this is perfectly working in all cases I must disappoint you. There are subtle things going on when static fields are initialized by the CLR. This implementation is thread safe because static fields are initialized only once by the CLR before any method can be called or a static field can be accessed from the outside. More details can be found inside the ECMA-334 C# Language Specification and ECMA-335 Common Language Infrastructure (CLI). Now comes the tricky part. When do you think will be LogWriter instance be instantiated? To answer this question let's have a look at two nearly identical Singletons.

Show more ...

The Evolution of Languages

Alois Kraus — Sat, 24 Jun 2006 02:19:00 GMT

Ron Garet did write about his personal experiences at NASAs JPL and how Lisp did perform in outer world (deep space) AI robotic projects. He was kinda disappointed by the way Lisp was abandoned in favor of C and Java because of C-Lisp integration problems. I think it would be interesting to have a look at todays programming languages and their possible future.
The programming languages differ in many aspects such as

Used Programming Paradigm
Standard Libraries
Syntax

Today the most widely used languages are imperative programming languages. With Microsoft pushing .NET the market for general purpose programming languages is consolidating. Thats a good sign. Why would I need more than one programming language at all? They are different but why are C/C++, Java and C# such a big success while others also well designed ones (e.g. Lisp) are not? The first thing that comes to my mind is ease of use and the effort to become good at it. Java makes it so easy that some think it has made programming too easy. Perhaps Java is todays BASIC to which we soon will point with the finger and say: "Java has no goto statement. Thats a bad thing". Because it is so easy to learn it is one of the first languages that our kids will be exposed to. That means that the programing language world is dominated by imperative languages with a virtual machine inside it for the next decades. I hope this will result in an ueber language with a basic syntax for all common things (why do I need to learn again and again how to loop over an string array in another language?) and a high quality Base Class Library which can be accessed by any language that does build on top on the used VM. While the syntax already does look similar of C/C++, C# and Java the used programming paradigm is the same. With the introduction of LINQ, which brings many constructs formerly only seen in functional programming language to the masses, different programming paradigms are supported within the very same language in a natural way.
The convergence of languages which drives us to one syntax, one Base Library and support for every Programming Paradigm will give us the flexibility we need to build even more complex systems in the future. That does sound too good to be true. But what if I need a hand optimized assembly routine for certain constructs? Well you can program today already low level assembly code with managed languages like C#. By analyzing how you need to arrange your looping constructs you can influence the JITer (Just In Time Compiler) to spit out different machine instructions. Greg Young has put up a very nice article how you can basically program assembler by using C#. The ability to write performant code in a very high level language will be the key success factor for a future ueber language which does bring all together. The low level hackers can do their assembly thing and the Java kids have their easy to learn BASIC. To make this happen we will need much more flexibility at the JIT side. Let's suppose you want to write a sophisticated 3D Game using managed code. Since the CPU is not good enough for high speed rendering we could annotate portions of our code to be optimized for the GPU (Graphics Processing Unit) which will execute there. Imagine the possibilities you would have if you could write things like this:

[JITWith("GraphicsCard_GPU_Module")]
void 3DProcessing()
{
foreach(Point p in Points) // This code is executed on your graphics card!
{
Draw(p);
}
}

By the possibility to extend the JIT compiler we do not have any managed unmanaged transition in our high level code anymore. I believe that the language war is not only fought with syntax and libraries but also at the JIT level. Perhaps we have in some years one common language syntax and many JIT extensions to create highly optimized domain specific code.
To have only one syntax is surely whisful thinking (but I like the idea) as it turns out that many complicated (read algorithmic) problems are not easy to implement with mainstream languages because unorthodox solutions will usually collide with good programming practices and the syntax. A smart syntax that is very expressive and does not impose much structure to your program (like enforcing OO) does attract people which write algorithms. So far I have (please correct me if I am wrong) seen the functional programming paradigm mainly in the field of:

Parsers
Compilers
Algorithms

The less structured syntax elements a language demands the easier it is to change the implementation of an algorithm totally because the structure of your algorithm does dictate the overall structure of your code and not the syntax of the language (e.g. C++ by enforcing OO where it does simply not fit) . In a highly flexible development environment (e.g. robotic research) where radically new ideas are constantly integrated into your code base the functional approach does work very good. But the lack of syntactic structure makes it hard to coordinate the work of many people which do usually work on a small component of a large project. This is where information hiding the and the success of Object Oriented Design comes into the game which brings us nearer to the goal of truly reusable components. Languages emerged to help us to support OO design intrinsically by enforcing it inside their syntax (C++, Java, C#, ...) . With Software Product Lines an old idea has made it into daily business that does create reusable components which are assembled into a commercial product. To make the product line idea work you need common stable interfaces which do not change too much over time. This requires many syntactic elements that hinder your creativity but help to enforce corporate policy at source code level (e.g. Java). No big mental abilities are needed for this, only a certain degree of discipline and the ability to judge where you should cut your program into smaller components to keep it maintainable. From this point of view I think we should be able to do very well with only two general purpose programming languages where one does support the component fraction whereas the other does help the quick and dirty prototypers. DSL (Domain Specific Languages) could be integrated as LINQ already shows us today into both to get a solid syntactic foundation that could much longer than todays rapid language evolution. Michael Stal has posted a nice article about his point of view how DSLs could be integrated into general purpose programming languages.

Why .NET Tracing is not reliable

Alois Kraus — Tue, 13 Jun 2006 02:18:00 GMT

This time I wanted to write something about the .NET Framework that is solved not optimal of the .NET Framework. My unsuspecting victim is the well known System.Diagnostics.Trace class. Static classes have the big plus that they are easy to use but they can be very hard to maintain if you want to extend it (no derivation) and have a lifetime problem with the objects that are created inside it. The Trace facade is in my opinion a prominent example of how you should not design a reliable static facade class. It's has many features that are nice but it's at the same time a fine example of design that has lost its vision. It tries to do too many things at the same time without succeeding in the optimal solution for all the goals it might have. Tracing should be fast, right? The .NET Trace definitely not the fastest solution. Yes it's flexible and can be very good configured inside your App.config file. But if you have used the Microsoft Logging Application Block of Enterprise Library 2.0 once then you know how much farther design for flexibility can be done. But I must admit that the Entlib TraceListeners does derive from the .NET ones. Below is a +- table about the .NET Trace regarding Performance Reliability and Extensibility.
In the following article I will show you what reliability issues there are and how to solve them.

Performance	Reliability	Extensibility/ Ease of use
+ TraceSwitches allow to configure at run time what should be traced. - DefaultTraceListener uses OutputDebugString which is by far the slowest method to trace.	- I loose traces if I do not call Trace.Close/Flush from time to time. - Static Trace.Close is a mistake because I cannot really close something that is static. - A Trace.Write after a Trace.Close causes loss of all following traces in case of file based listeners. (+ Some Listeners do reopen itself after being closed. The File Listener can do this if you init it with a file name)	+ Dynamic Trace Switch Configuration + Dynamic Trace Listener Configuration - Other (Logging) solutions are far more flexible. - It is nearly impossible to write a program that does capture all traces even during application shutdown.

The most obvious reliability problem I did find is demonstrated with a very simple Console Application:

Show More ...

Functional Programming with F#/C#

Alois Kraus — Sun, 28 May 2006 23:30:00 GMT

After all this hype around C# 3.0 where we will get LINQ, lambda expressions and many other thing I thought that it would be useful to have a deeper look at functional programming languages like F#. At Microsoft quite many people are fond of the Functional Programming style which did influence the design of C# 3.0. Many language architects there have Haskell background which could explain the renaissance of these "old" concepts. So what's the deal with the functional approach? It does basically force you to think in a twisted way where the problem is straight solved by calling a function which does call another function and so on. Lets assume that you want to get the list of all members of all types which are contained in the assemblies that are currently loaded into your AppDomain (to look at the problem this way is the twisted part) you could write for example (in a functional language like F# this problem is very easy to solve):

// Assign allMembers the result of our "calculation"
let allMembers = System.AppDomain.CurrentDomain.GetAssemblies()
 |> Array.to_list |> List.map (fun a -> a.GetTypes()) |> Array.concat
 |> Array.to_list |> List.map (fun ty -> ty.GetMembers()) |> Array.concat

// Print them out 
do Array.foreach allMembers ( fun memberInfo -> printf "\n%s" memberInfo.Name)

If you want to understand what this little code fragment really does and how it can be rewritten I recommend the post from Robert. A more comprehensible version in C# code would look like this:

public List<MemberInfo> GetAllMembers()

{

Assembly [] allAssemblies = AppDomain.CurrentDomain.GetAssemblies();

List<Type> allTypes = new List<Type>();

foreach(Assembly assembly in allAssemblies)

{

allTypes.AddRange(assembly.GetTypes());

}

List<MemberInfo> allMembers = new List<MemberInfo>();

foreach( Type t in allTypes )

{

allMembers.AddRange( t.GetMembers() );

}

return allMembers;

}

Create Live Videos From Your Screen

Alois Kraus — Tue, 09 May 2006 01:28:00 GMT

Just recently I found a very good post how to create your own live demo video with Windows Media Encoder. John Howard did put on his blog a useful post about the preferred settings for a good encoding experience. The main thing is to get the small videos of your computer screen is that you need to set as resolution 800x600 and 16 bit color depth. This is also a very good choice if you want to share your video in a meeting room with a beamer since most of them do not support more than 800x600. After digging a bit further I did find out that I have installed on my Windows XP PC a movie editing software with since Service Pack 2. It is called Movie Maker and resides in your Program Files directory. I have installed so many programs on my PC without knowing anything about it! Scary ... Armed with these tools it is really easy to create a live demo for some presentation which will never fail. No more: "Hm it worked always before". What is even cooler that now everybody can take the demo home and watch the steps at the time he/she does need it. You do not have to do brain surgery to remember what the heck this guy did tell you three weeks ago in his demo to get this thing working. Just open the video and watch it again. This could really become a new trend since it is so easy to create a video demo this way. I found it much easier to show it live than to write (nobody reads it anyway) documentation and later people call you just to ask the question which was answered at page 95 of your famous (though unread) document. Next time I will send them the link to the demo video.

Performance Quiz: Fastest way to format time

Alois Kraus — Sun, 23 Apr 2006 22:55:00 GMT

We want a good performing application but it is hard to get it right. We all know how to format strings in a fast and efficient way don't we? Let me challenge you if anybody can beat my lightning fast fixed string formatting function. The rules are easy:

We want to format the content of DateTime structure into a string of the form: "08:44:45.423"
The performance measured is the time needed to return the formatted string when the DateTime structure is as input given.
.NET 2.0 and C# only.

For starters I publish here the class which measures the performance

public static class Stopper

{

public delegate string TestFunction(DateTime time);

static int myIterations;

public static int Iterations

{

get { return myIterations; }

set { myIterations = value; }

}

public static void CheckTime(string action, TestFunction func)

{

Stopwatch Watch = Stopwatch.StartNew();

DateTime curTime = DateTime.Now;

for (int i = 0; i < Iterations; i++)

{

func(curTime);

}

Watch.Stop();

Console.WriteLine("Action {0} did consume {1:F2} s",

action, Watch.ElapsedMilliseconds / 1000.0f);

}

The test functions are executed in the Tests function describing itself during their execution:

public void Tests()

{

Stopper.CheckTime("Format with String.Format", FormatWithStringFormat);

Stopper.CheckTime("Format with StringBuilder.AppendFormat", FormatWithStringBuilder);

}

A possible solution to the String format question is the good old String.Format:

public string FormatWithStringFormat(DateTime time)

{

return String.Format("{0:D2}:{1:D2}:{2:D2}.{3:D3}",

time.Hour, time.Minute, time.Second, time.Millisecond);

}

Or is StringBuilder the best one?

StringBuilder sb = new StringBuilder();

public string FormatWithStringBuilder(DateTime time)

{

sb.Length = 0; // reset string builder buffer

sb.AppendFormat("{0:D2}:{1:D2}:{2:D2}.{3:D3}",

time.Hour, time.Minute, time.Second, time.Millisecond);

return sb.ToString();

}

Static inits do not count and are not part of the quiz. I will post the code of the fastest(?) string time formatting function in a few days. Perhaps somebody out there can beat my function. This are the results at my home PC (P4 3.0 GHz 1 GB RAM) after doing 2 million string formats. As you can see the String.Format is not the best option to choose ...

Formatting Algo	Elapsed Time in s	Specialities
String.Format	5,54
StringBuilder.AppendFormat	4,92
Bruce ToString	3,36
Bruce FastFormat	1,81
My Best Function So Far	1,75	Unsafe Code
FastFormat4 by Greg Young	1,15	Unsafe Code
NewFastestFormat	1,08	Unsafe Code
DansFormat	0,83	Unsafe Code
FormatFastSafe by Jeffrey	0,53
FastFormat6 by Jeffrey Sax	0,42	Unsafe Code
NewFastestStringFormatting	0,43	Unsafe Code
FormatFast7 by Greg Young	0,33	Unsafe Code, Breakes the CLR string handling!

Bruce did submit some samples which perform already very good. His FastFormat is looks very promising. By realizing that you can use a fixed char array and also knowing that the string constructor can take an char array we can get an impressive performance boost by a factor three compared to the most obvious solution. But the very creative solution from Greg Young did really surprise me. He did:

Realize that the DateTime get properties do not simply return an int but do some calculations on a 64 bit value. Repeated access to the get properties do cost therefore quite a lot and can be minimized by doing it yourself and cache the calculated values for hour, minute, second and millisecond.
Use unsafe access to the array (My Best Function does this already but the effect is quite small).

The result is a 40% speed improvement. Very impressive Greg. There proves once again that there is always somebody around who is smarter than you. But all is not all lost. I did even optimize Gregs function a little bit and made it more readable. I strongly believe that reflector had something to do with his success. Here is the code for my NewFastFormat function which is by the way much more comprehensible:

public unsafe string NewFastestFormat(DateTime time)

{

fixed (char* charP = &FixedCharArray[0]) // Get rid of bounds checking

{

int hour = time.Hour; // Cache property values

int minute = time.Minute;

int second = time.Second;

int ms = time.Millisecond;

// hour

charP[0] = (Char)('0' + hour / 10);

charP[1] = (Char)('0' + hour % 10);

// minute

charP[3] = (Char)('0' + minute / 10);

charP[4] = (Char)('0' + minute % 10);

// seconds

charP[6] = (Char)('0' + second / 10);

charP[7] = (Char)('0' + second % 10);

// miliseconds

charP[9] = (Char)('0' + ms / 100);

charP[10] = (Char)('0' + ms % 100 / 10);

charP[11] = (Char)('0' + ms % 10);

}

return new String(FixedCharArray);

}

My last trick I came up with is that I do not need to recreate the array at every call. And that I can initialize the ": : ." characters only once in the ctor of my class. This squeezes the last(?) bit of performance out of it. I can only say: WOW. I did not think that this big improvement was possible. After some test runs with and without unsafe array access it turned out I was not able to measure any difference! Strange but true which proves that you should not trust other peoples claims about performance. I am surprised that nobody did try it with a generic List<Char>. Many say that generics are faster but the string constructor does not take such a thing. So we have to convert it back to a char array which kills performance. Another interesting thing is that the StringBuilder index operator [] is significantly slower than the char array indexer. Now we can safely say that we have here the fastest time to string formatting function known to C# programmers until somebody finds an even faster solution.
Update: There is somebody out there who did find an even faster solution: Jeffrey Sax is obviously a real number crack and plays dirty tricks by creeping around expensive divisions by turning them into multiplications with a following bit shift. I had numerical maths at university during my studies of nuclear physics but this is really tough stuff. To push my ego up I must say that he does this for a living at Extreme Optimizations, which explains why he did find such a dam good function. He did make it by a factor 2.6 faster that our previously fastest function! This means that we posses now a string formatting function which is over 14 times faster than our original String.Format. Unbelievable. Now I think we have reached a limit where we have used all our tricks that are imaginable. I do not expect that there is room for another big improvement such as this. But one never knows.... As usual I did a little reformat and added some comments to make it easier for normal people to get the idea:

unsafe string NewFastestStringFormatting(DateTime time)

{

fixed (char* charP = &FixedCharArray[0])

{

// Calculate values by getting the ms values first and do then

// shave off the hour minute and second values with multiplications

// and bit shifts instead of simple but expensive divisions.

long ticks = time.Ticks;

int ms = (int)((ticks / 10000) % 86400000); // Get daytime in ms which does fit into an int

int hour = (int)(Math.BigMul(ms >> 7, 9773437) >> 38); // well ... it works

ms -= 3600000 * hour;

int minute = (int)((Math.BigMul(ms >> 5, 2290650)) >> 32);

ms -= 60000 * minute;

int second = ((ms >> 3) * 67109) >> 23;

ms -= 1000 * second;

// Hour

int temp = (hour * 13) >> 7; // 13/128 is nearly the same as /10 for values up to 65

charP[0] = (char)(temp + '0');

charP[1] = (char)(hour - 10 * temp + '0'); // Do subtract to get remainder instead of doing % 10

// Minute

temp = (minute * 13) >> 7; // 13/128 is nearly the same as /10 for values up to 65

charP[3] = (char)(temp + '0');

charP[4] = (char)(minute - 10 * temp + '0'); // Do subtract to get remainder instead of doing % 10

// Second

temp = (second * 13) >> 7; // 13/128 is nearly the same as /10 for values up to 65

charP[6] = (char)(temp + '0');

charP[7] = (char)(second - 10 * temp + '0');

// Milisecond

temp = (ms * 41) >> 12; // 41/4096 is nearly the same as /100

charP[9] = (char)(temp + '0');

ms -= 100 * temp;

temp = (ms * 205) >> 11; // 205/2048 is nearly the same as /10

charP[10] = (char)(temp + '0');

ms -= 10 * temp;

charP[11] = (char)(ms + '0');

}

return new String(FixedCharArray);

}

As you can see this is not exactly rocket science it is worse than that. You have to be very careful to not get biten by a rounding error due to a too big value. The initial version of Jeffrey did have a small error which caused for some ms values strange characters. Really great work Jeffrey. For educational purposes I do also post there the fastest format function which does no unsafe tricks. Credits for FormatFastSafe go also to Jeffrey. If still more solutions are required to be considered I am forced to rate the solutions in categories. Fastest function ever is Gregs solution but you would not use this function really since it does break the CLR promise that string objects are immutable. The fastest function which does contain unsafe code and works correctly was sent by Jeffry. And the fastest (correct working) function which uses no unsafe tricks was also sent by Jeffrey. The fastest safe function is this one:

private string FormatFastSafe(DateTime time)

{

long ticks = time.Ticks;

int n1 = (int)(ticks >> 32);

int n2 = (int)ticks;

if (n2 < 0)

n1++;

ticks = (((Math.BigMul(429497, n2) - (int)(Math.BigMul(1161359156, n2) >> 32) - Math.BigMul(1161359156, n1)) >> 32))

+ Math.BigMul(n1, 429497);

n1 = (int)(ticks >> 32);

n2 = (int)ticks;

if (n2 < 0)

n1++;

int q = n1 * 50 + ((50 * (n2 >> 16)) >> 16) - (int)(System.Math.BigMul(1244382467, n1) >> 32) - 1;

int r = (int)(ticks - System.Math.BigMul(q, 86400000));

if (r > 86400000)

r -= 86400000;

int unit = (int)(Math.BigMul(r >> 7, 9773437) >> 32) >> 6;

n2 = (unit * 13) >> 7;

n1 = r - 3600000 * unit;

FixedCharArray[0] = ((char)(n2 + '0'));

FixedCharArray[1] = ((char)(unit - 10 * n2 + '0'));

unit = (int)((Math.BigMul(n1 >> 5, 2290650)) >> 32);

n1 -= 60000 * unit;

n2 = (unit * 13) >> 7;

FixedCharArray[3] = ((char)(n2 + '0'));

FixedCharArray[4] = ((char)(unit - 10 * n2 + '0'));

unit = ((n1 >> 3) * 67109) >> 23;

n1 -= 1000 * unit;

n2 = (unit * 13) >> 7;

FixedCharArray[6] = ((char)(n2 + '0'));

FixedCharArray[7] = ((char)(unit - 10 * n2 + '0'));

n2 = (n1 * 41) >> 12;

FixedCharArray[9] = ((char)(n2 + '0'));

n1 -= 100 * n2;

n2 = (n1 * 205) >> 11;

FixedCharArray[10] = ((char)(n2 + '0'));

FixedCharArray[11] = ((char)(n1 - 10 * n2 + '0'));

return FixedCharArray.ToString();

}

This contest was really fun and I did not expect to get so many good solutions. Thanks to everyone who did submit sample code. The code presented here is a very good example of highly optimized C# code which (I guess) cannot be found anywhere else on the net. I hope it does help other programmers to see where they burn CPU cycles and by what means you can solve a performance problem.

Assembly.LoadFrom demystified

Alois Kraus — Wed, 05 Apr 2006 02:39:00 GMT

Rules are meant to be bro.. ahem expanded. Especially if everybody tells you not to do so ;-). The Assembly Loader Rules are for example a very strict set of rules. One rule of the loader is that you cannot load assemblies from directories above the directory where the executable is located. This can cause some headaches if you want to share some assemblies across applications which are not installed inside the GAC. You can bypass this restriction by using Assembly.LoadFrom. A very good article about the how and when of Assembly.LoadFrom can be found at GotDotNet. If you have to deal with plugin architectures and loosely coupled components you will soon end up in a situation where you need to expand the default assembly loader rules. You can configure the loader to some extent via your App.Config file but you cannot configure a probing path for directories above your application binary location. Lets suppose you have an application lets call it App.exe which references ClassLibrary1.Dll which in turn references ClassLibrary2.dll. Now you have to call Assembly.LoadFrom for ClassLibrary1.dll for some reason (e.g. it is a configured plugin). I have drawn the dependencies in the diagram below

Show More ....

Call Properties through delegates and pass them by Reference

Alois Kraus — Sat, 11 Feb 2006 02:39:00 GMT

After reading Mike Stalls Debugging Blog I was wondering if it is really impossible to pass property values by reference:
    void PassIntByRef(ref int x)
    ...
    PassIntByRef(ref MyProperty); <-- this is illegal in C#.
Another restriction which is often annoying is that you cannot have property delegates.

    class PropertyDelegation
    {
        public delegate int ReturnValueDelegate();
 
        ReturnValueDelegate ValueGetter;
 
        int x = 0;
        public int X  // This is our beloved property we care so much about.
        {
            get { return x; }
            set { x = value; }
        }
 
        public PropertyDelegation()
        {
            ValueGetter = X;      // Illegal in C#
            ValueGetter = GetX;   // Legal code
        }
        int GetX()
        {
            return X;
        }
   }

The above workaround with the GetX function is not really nice but it works and I wondered if there is no other way to do this. After 10 minutes thinking about this I did realize that both problems could be solved by a reference type which provides the needed functionality.

The new type should provide a getter and setter for the wrapped property.
Expose Get and Set functions to attach delegates to it
Provide a as much type safety as possible.

Lets look what the C# compiler does produce for a Property with a getter and setter. Using Reflector we quickly see what really happens:

Hm the property is in reality a set of two methods. If we get our hands on both methods we can create a delegate from it and call them by our selfs inside our wrapper class (reference type object). Extensions to compiled types at run time calls for the reflection API. In our case the PropertyInfo class provides all information we want. The result of this is my new class PropertyInvoker which fulfills all three requirements.

    public class PropertyInvoker
    {
        private PropertyInfo propInfo;
        private object obj;
 
        public PropertyInvoker(string PropertyName, object o)
        {
            obj = o;
            propInfo = o.GetType().GetProperty(PropertyName);
        }
 
        public T Property
        {
            get
            {
                return (T) propInfo.GetValue(obj, null);
            }
 
            set
            {
                propInfo.SetValue(obj, value, null);
            }
        }
 
        public T GetProperty()
        {
            return Property;
        }
 
        public void SetProperty(T value)
        {
            Property = value;
        }
    }

This little helper allows us to create a delegate function from a property:

        public delegate int ReturnValueDelegate();
        ReturnValueDelegate ValueGetter;

        // Create delegate around the get property of X
        ValueGetter = new ReturnValueDelegate(new PropertyInvoker<int>("X", this).GetProperty);
        int realval = ValueGetter.Invoke(); // Call get property with our new invoker

Properties cannot be really passed by reference but at least we did come quite close to our initial goal:

        private void Add(PropertyInvoker<int> value)
        {
            value.Property++;
        }

        // Pass property by reference inside wrapped type
        Add(new PropertyInvoker<int>("X", this));
 

Ok this looks nice but how about performance? Reflection is known to be slow after all. The good news is that on my P4 3.0 GHz PC at home I can call value.Property++ over 300 000 times/s. This is quite good for an indirect function call via reflection. The bad news is the native property is much better. To be precise about several hundred times faster. Why? I think the JITer is very good at detecting this set_X and get_X methods in my code and does simply inline the calls. Against inlined function calls my PropertyInvoker has clearly no chance since at runtime there is no method call due to god work of the JITer. To check this assumption and to prevent such an unfair "optimization"I decorated the getter and setter of my property with the [MethodImpl(MethodImplOptions.NoInlining)] attribute to prevent inlining. After that I got "only" a performance penalty of about 200 times slower than the native method call. To sum it up: My method is much slower as a native property (or wrapped function call) but still far away from being slow in absolute terms. If this little class does help you to write nicer and cleaner code give it a try.