Unit Testing

VS 2008 crashes when running unit tests - VSPERF?

Tim Huffam — Tue, 26 Aug 2008 21:36:21 GMT

Sometimes Visual Studio 2008 (Team System Developer Edition - with Team Explorer and TFS Power Tools) crashes (disappearing completely) when running unit tests.

I have not yet tried SP1 for VS2008 - will hopefully try this in the next week or two.

Not really sure what causes this - but have noticed that the following error is written to the windows event log prior to the crash (not at the same time).

Event Type: Error
Event Source: VSPERF
Event Category: None
Event ID: 1406
Date:  22/08/2008
Time:  16:27:18
User:  N/A
Computer: MYPC
Description:
There was an error connecting to the Logger Engine while registering C:\Dev\DEV\Release1\Source\MyWebProject\bin\MyWebProject.dll for coverage.

Initially I thought it was a particular test or peice of target code that caused it, but found that the crash occured at seemingly random moments ie I tried running all tests and it would crash at test number 259 (out of 264), then tried running the first 100 tests and it crashed - then tried the last 20 tests and it crashed- even tried running 1 test and it crashed. This was the worst case of this problem - over the past few weeks it (VS crashing/disappearing) would only happen every now and then.

The only work around I've found is to replace or remove the workspace and start again...

If you have pending changes, shelve them first.
Using Source Control Explorer, remove the workspace.
Close the solution in VS.
Within a VS command prompt run the following command to remove all workspaces (if you have many workspaces and only want to remove the one workspace then specify it's name instead of the asterisk used below):
tf workspaces /remove:*
Use the Source Control Explorer to recreate a new workspace.
Note: if you have scripts/macros that reference your original workspace source dir, then it would pay to use the same dir - if you do, make sure you delete the contents of this dir before reassigning it to the new workspace.

HTH
Tim

Checking ExpectedException and is thrown by specified type

Tim Huffam — Fri, 06 Jun 2008 04:19:22 GMT

Using the VS2008 unit testing framework, if you want to check for an expected exception, sometimes your test will pass because the exception was raised by another line of code - after the line you were specifically trying to test.

To resolve this you have to manually check for the expected exception - using a try-catch block.

However there is a catch (excuse the pun)....

Microsoft.VisualStudio.TestTools.UnitTesting.Assert.IsInstanceOfType() will not fail when types are different as long as the actual value is of a type within the inheritance hierarchy of the expected type.

Eg the following assertion will not fail if ex is of type ArgumentNullException - as it inherts from ArgumentException.

Assert.IsInstanceOfType(ex, typeof(ArgumentException));

One solution is to create your own assertion method eg:

public static void ExpectedExceptionThrownByType(Type expectedExceptionType, Exception actualException, Type expectedThrownByType)
{
Assert.AreEqual(expectedExceptionType, actualException.GetType());
Assert.AreEqual(actualException.TargetSite.DeclaringType, expectedThrownByType, "Expected exception to be thrown by type " + expectedThrownByType.ToString() + " but was " + actualException.TargetSite.DeclaringType.ToString());
}
Note the first assertion uses AreEqual instead of IsInstanceOf() - this makes sure the exception is of a specific type (rather than any one up the inheritance heirarchy).

Unit testing with NUnit and member accessibility - and the InternalsVisibleTo attribute

Tim Huffam — Wed, 29 Aug 2007 16:39:26 GMT

One of the annoying things about unit testing, is that not all members of the target code class are accessibly to the test fixtures.

There are limited options to work around this (as you really don’t want to put the unit tests within the code assembly).

In the past, the usual method has been to open up the access levels of the code class members. This isn’t ideal from a best practices and security point of view.

However, as of .Net 2.0 a better method has been introduced. This is to use the InternalsVisibleTo attribute on the code assembly. This makes any code members with an accessibility level of internal or above (internal, protected internal and public) available to the specified assemblies.

To use this attribute add the following line to the AssemblyInfo.cs file of the code assembly:

[assembly: InternalsVisibleTo("MyAssembly.UnitTests")]

where MyAssembly.UnitTests is the unit test assembly name.

Tim

.Net Error: Microsoft.Practices.EnterpriseLibrary.ExceptionHandling.ExceptionHandlingException : The type '... .LoggingExceptionHandler..' cannot be resolved.

Tim Huffam — Sun, 26 Aug 2007 16:24:19 GMT

Here's another problem I had when trying to get the Microsoft Enterprise Library - exception logging working.

The following error was caused because I did not add references to all the assemblies used.

MyApp.UnitTests.BuildHelperTests.DBConnectionString_Get : Microsoft.Practices.EnterpriseLibrary.ExceptionHandling.ExceptionHandlingException : The type 'Microsoft.Practices.EnterpriseLibrary.ExceptionHandling.Logging.LoggingExceptionHandler, Microsoft.Practices.EnterpriseLibrary.ExceptionHandling.Logging, Version=3.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' cannot be resolved. Please verify the spelling is correct or that the full type name is provided.
----> System.ArgumentException : The type 'Microsoft.Practices.EnterpriseLibrary.ExceptionHandling.Logging.LoggingExceptionHandler, Microsoft.Practices.EnterpriseLibrary.ExceptionHandling.Logging, Version=3.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' cannot be resolved. Please verify the spelling is correct or that the full type name is provided.

In this case I needed to add reference to Microsoft.Practices.EnterpriseLibrary.ExceptionHandling.Logging.dll.

Also, when unit testing my app, VS did not copy this (and a couple of other assemblies) from the original code project to the unit test project - so the solution here was to add a post build event to copy them across to the unit test target (bin/debug) dir.

HTH

Tim

NUnit not working properly - cannot find TestFixtures (Reason: Has no TestFixtures)/getting Visual Studio to automatically run NUnit

Tim Huffam — Wed, 01 Aug 2007 14:23:06 GMT

When trying to run NUnit against an assembly, it loads the assembly fine, but when you run it, it cannot find any tests (even though the assembly contains many tests).

Under the 'Tests Not Run' tab, if you expand the treeview, it states: "Reason: Has no TestFixtures".

This problem is caused when NUnit does not have adequate rights to the assembly. In my case it was because my project was on a network drive.

A simple solution to this is to add a post build event in Visual Studio to copy the assembly files from the network drive onto a local drive (ie in a temp dir), then run NUnit against this temp copy.

1. Within VS, open the project properties.

2. Go to the Build Events tab and enter the following 'Post-build event command line':

del /q c:\temp\nunit\*.*
copy "$(TargetDir)*.*" c:\temp\nunit

3. Go to the 'Debug' tab and enter the following details:

Start external program: C:\Program Files\NUnit 2.4.1\bin\nunit.exe

Command line arguments: "c:\temp\nunit\MyProject.dll" /run

Now when you click run on your solution it will start NUnit, load the assembly and run the tests automatically.

Running NUnit against multiple versions of .NET (1.1 and 2.0) from Visual Studio (VS.NET 2003 and VS2005)

Tim Huffam — Wed, 29 Mar 2006 20:47:00 GMT

This is not the prettiest solution - but is the only way I could easily get it all working - so I could click the Run button from VS to startup NUnit and run my tests project.

As a primer, read James Newkirks article on this (he explains the config files a bit more than I do).

Firstly to get NUnit to work with .NET 2.0 assemblies you need to have this in your nunit config file (nunit-gui.exe.config):

If you include a line referencing v1.1 then it'll use 1.1.

So this means to run both 2.0 and 1.1 we need 2 config files.

The 1.1 config file has this startup section:

Unfortuntely I couldnt see any obvious way to tell NUnit to point to a specific config file (there is /config option but I believe that is for configurations within NUnit - not which app config file to use). So to get around this there are 2 options:

Use 2 config files and use a batch script to rename the appropriate one to nunit-gui.exe.config prior to starting Nunit.
Use 2 config files and 2 executable files (named to match the config files).

I chose 2. because it removes any ambiguitiy. So what I have are the following files in the Nunit bin dir:

nunit-gui_1.1.exe
nunit-gui_1.1.exe.config
nunit-gui_2.0.exe
nunit-gui_2.0.exe.config

Running the exe's will start NUnit with the corresponding .NET runtime version.

Now to get it working from VS was the next problem...

For some reason starting NUnit for 2.0 from VS2005 does not work.

The workaround (dont have time to figure out the real cause) is to create a 1.1 console app that executes another program - like a batch script (I tried a batch script but that had complications too). Here's the code:

public static void Main(string[] args)
{
    if(args.Length==0)
    {
        Console.WriteLine("Error: No command specified.");
        Console.WriteLine("Syntax: CmdRunner.exe [ ...]");
        Console.WriteLine("Note must be the full path to the executable. Parameters are optional.");
        return;
    }
    string paramsToPass = "";
    for(int i=1;i    {
    if(paramsToPass!="")
        paramsToPass+=",";
    paramsToPass += args[i];
    }
    Console.WriteLine("About to run: " + args[0]);
    Console.WriteLine("Passing params: " + (paramsToPass==""?"":paramsToPass));
    try
    {
        System.Diagnostics.Process.Start(args[0], paramsToPass);
    }
    catch(Exception ex)
    {
        Console.WriteLine("Error occured trying to run: " + args[0]);
        Console.WriteLine("Exception: " + ex.Message);
    }
}

I called this CmdRunner.exe.

Next problem was that this didnt like spaces in the parameters passed to it (these were in my project directory structure). So modify your path to use the old 8.3 notation (to find out what this is do a "dir /x" at the command prompt).

Lastly, setup VS to run this when you click on Run:

VS.NET 2003:

Under project properties select Debug.
Set Debug Mode to Program
Set Start Application to the CmdRunner path eg: O:\Visual Studio Projects\CmdRunner\bin\Debug\CmdRunner.exe
Set Command Line Arguments to the appropriate NUnit version plus the assembly dll. It is critical that these contain no spaces. Also, if you want NUnit to automatically run the tests upon startup, then the /run argument needs to be included in the second argument string (this has one space before it). Note these must be enclosed in quotes eg:
"C:\Progra~1\NUNIT2~1.2\bin\nunit-gui_1.1.exe" "C:\Dev\VSSWRK\Ecommerce\DotnetVarious\DRKW~1.DIR\Tests\bin\Debug\Tests.dll /run"

VS2005:

Under project properties select Debugging tab.
Set Start Action to 'Start external program' and set it's path to the CmdRunner path. Because this 1.1 program is CLR compatible with 2.0 there's no problems running it from here) eg: O:\Visual Studio Projects\CmdRunner\bin\Debug\CmdRunner.exe
Set Command Line Arguments to the appropriate NUnit version plus the assembly dll. It is critical that these contain no spaces. Also, if you want NUnit to automatically run the tests upon startup, then the /run argument needs to be included in the second argument string (this has one space before it). Note these must be enclosed in quotes eg:
"C:\Progra~1\NUNIT2~1.2\bin\nunit-gui_2.0.exe" "C:\Dev\VSSWRK\Ecommerce\DotnetVarious\DRKWDI~1.1\Net2.0Tests\bin\Debug\Net2.0Tests.dll /run"

And that's it. Good luck.

HTH

Tim

System.BadImageFormatException : The format of the file ‘TestLibrary’ is invalid. NUnit on VS 2005 / .NET 2.0

Tim Huffam — Fri, 24 Mar 2006 00:37:00 GMT

A thousand thank you's to James for this post: