Testing & Debugging

How to Use Extension Methods and Lambda Expressions to Write Elegant Unit Tests

Chris Falter — Wed, 30 Dec 2009 21:23:54 GMT

Suppose you have to write a test to verify that an edit form will throw a certain exception under certain conditions. If you don't use VB 9 features, you might end up with this effusion of verbosity:

Try

            editForm.Save(MyDataContext)

            Throw New AssertFailedException("Save should have thrown an exception")

        Catch ex As Exception

            If Not (ex.GetType().Equals(GetType(BusinessOperationException))) Then

                Dim errorMessage = String.Format("Save should should have thrown BusinessOperationException; threw {0} instead", ex.GetType())

                Throw New AssertFailedException(errorMessage)

            End If

            Dim bopException = DirectCast(ex, BusinessOperationException)

            Assert.IsTrue(bopException.DataFormErrorInfo.SQLErrorInfo.ErrorMessage.Contains("ERR_MEMBERSPECIFIEDMEMBERS_INVALID"))

        End Try

Of course, you could make the code a little simpler by not handling any exceptions of the wrong type, which would implicitly cause the test to fail if an exception of the wrong type is thrown:

Try

             editForm.Save(MyDataContext)

             Throw New AssertFailedException("Save should have thrown an exception")

         Catch ex As BusinessOperationException

             ' exceptions of other types are not handled

             Assert.IsTrue(ex.DataFormErrorInfo.SQLErrorInfo.ErrorMessage.Contains("ERR_MEMBERSPECIFIEDMEMBERS_INVALID"))

         End Try

While this is better, the code is not without problems:

The reader of this code has to infer that exceptions of a type other than BusinessOperationException will fail cause a test failure. Yes, you can mitigate this by including a helpful comment--if you remember. I'd rather not have to remember.
The intention of the test is buried inside the mechanics of the try/catch block.
The assertion about the state of the BusinessOperationException instance is not easy to follow, since the reader of the code has to walk a lengthy property hierarchy (DataFormErrorInfo => SQLErrorInfo => ErrorMessage). This is a glaring violation of the Law of Demeter. In addition, it is not self-evident from reading the code that DataFormErrorInfo.SQLErrorInfo.ErrorMessage would naturally be the location where one would find the argument to a SQL raiserror call.

Prior to VB 9 it would have been very difficult to write this code with a smoother syntax due to the limitations of the language. But now we can solve these problems by using lambda expressions and extension methods, as follows:

        Dim saveForm As Action = Function() Me.Save(editForm)

        saveForm.ShouldThrow(Of BusinessOperationException).Where(Function(e) e.DBExceptionCode.Contains("ERR_MEMBERSPECIFIEDMEMBERS_INVALID"))

Here the fact that the form's .Save() method should throw a BusinessOperationException with a "ERR_MEMBERSPECIFIEDMEMBERS_INVALID" exception code walks up to the reader and punches him in the face. And that's what we want all of our code to do; pugnacious code is good code! Moreover, the ShouldThrow() and Where() methods will emit useful, detailed messages for any error conditions. Let's see how VB 9 helps us to work this magic.

Extension Methods

An extension method acts as if it belongs to a type; thus you may create an extension method for a type even if you do not possess the type's source code. Without extension methods, your only choice is to create a shared helper method that takes a type instance as a parameter. For example, in earlier VB versions you could have written a shared Throws() method that takes an Action delegate as a parameter. However, the extension method has greater clarity and readability:

Extension Methods vs. Shared/Static Methods
Coding Technique	Extension Method	Shared Method
Syntax	Subject + Verb + arguments/modifiers	Verb + list of arguments
Example	saveForm.ShouldThrow(Of BusinessOperationException)	Throws(Of BusinessOperationException)(saveForm)
Readability	Close to standard English - very readable	Completely at odds with English grammar. And when there are many arguments, which is the subject and which are modifiers?

Ultimately an extension method is syntactical sugar; then again, a little sugar can help you bake some wonderful recipes. So let's check it out.

In the solution code above, the generic ShouldThrow(Of TException) method is as an extension method to the .NET Action delegate. Let me pause for a moment to let that concept sink in; we are writing code that extends a built-in .NET type. And that type isn't even a class; it's a delegate! Clearly, extension methods open up a new realm of possibilities to us. Decorating a standard shared method with the System.Runtime.CompilerServices.ExtensionAttribute turns it into a method that behaves as if it were a member of the type of its first argument. Thus the first argument to the ShouldThrow method in Blackbaud.AppFx.UnitTesting/AssertExtensions.vb is of type Action delegate:

    <Extension()> _

    Public Function ShouldThrow(Of TException As Exception)(ByVal workToDo As Action) As TException

        Dim errorMessage As New StringBuilder("An exception of type ")

        errorMessage.Append(GetType(TException))

        errorMessage.Append(" should have been thrown, but wasn't.  Test method info follows...")

        errorMessage.AppendLine()

        errorMessage.AppendCallingMethodInfo()

        Return ShouldThrow(Of TException)(workToDo, errorMessage.ToString())

    End Function

    <Extension()> _

    Public Function ShouldThrow(Of TException As Exception)(ByVal workToDo As Action, ByVal errorMessage As String) As TException

Try

            workToDo.Invoke()

            Throw New AssertFailedException(errorMessage)

        Catch ex As Exception

            If (GetType(TException).Equals(ex.GetType)) Then

                Return ex

            End If

            Throw

        End Try

    End Function

The .ShouldThrow() method encapsulates all of the try/catch logic that had obfuscated the earlier versions of our test code. Just specify the type of the exception that should be thrown when you call it, and it will do one of three things:

return the thrown exception of the desired type,
rethrow a thrown exception of the wrong type, or
throw its own exception if an exception is not thrown.

In VB an extension method must belong to a module. (C# uses a static class.) After you compile the module containing the ShouldThrow() method, the .NET languages will allow you to use it as if it were actually a member of the Action delegate type. An extension method does have a limitation that true members of a type do not have: it cannot access the non-public members of the type. Alas, even magic has limits.

Another nice aspect of writing an extension method is that it will even show up in Intellisense! Already the advantage of the the extension method is clear; if you were using an older-style shared method, Intellisense would not list it as a candidate for use with the Action delegate saveAction.

While an overload of the ShouldThrow() method is available for those who wish to provide a custom error message, the default (parameterless) overload uses the AppendCallingMethodInfo() method to incorporate the name, source file, and line number of the calling test method into a useful error message. I recommend using the default overload of ShouldThrow() for most situations, since it produces a very useful error message while keeping your code's syntax as simple as possible. The alert reader will note that AppendCallingMethodInfo() is implemented as an extension method to the StringBuilder class; the curious may read through AssertExtensions.vb source file to see how the info is extracted from a stack trace.

The AssertExtensions module also contains the .Where() extension method overloads which, as we see from their first parameter, can be called on a generic exception.

    <Extension()> _

    Public Function Where(Of TException As Exception) _

    (ByVal ex As TException, ByVal predicate As Func(Of TException, Boolean)) _

    As Boolean

        Dim errorMessage As New StringBuilder("The " + GetType(TException).ToString() + " did not satisfy the Where predicate.  Test method info follows...")

        errorMessage.AppendLine()

        errorMessage.AppendCallingMethodInfo()

        Return Where(ex, predicate, errorMessage.ToString())

    End Function

    <Extension()> _

    Public Function Where(Of TException As Exception) _

    (ByVal ex As TException, ByVal predicate As Func(Of TException, Boolean), ByVal errorMessage As String) _

    As Boolean

        If Predicate(ex) Then

            Return True

        Else

            Throw New AssertFailedException(errorMessage)

        End If

    End Function

The Func keyword, new in VB 9, denotes a generic function delegate whose signature is determined by the types of the parameters. The last parameter is the return type of the delegate, and the leading parameters before it (if any) specify the types of the input parameters. Applying this syntax to our code, we see that the parameter ByVal predicate As Func(Of TException, Boolean) denotes a delegate for a function that takes a generic TException argument and returns Boolean. The Where() method invokes the delegate and takes care of throwing an AssertFailedException with an appropriate message if the delegate returns False. You will recall that the sample code used a lambda expression to create the delegate invoked by Where().

Finally, the ability to write an extension method allows us to obey the Law of Demeter, even though we cannot modify the BBSolder-generated BusinessOperationException class. In the unit test file, just define an extension method that traverses the properties:

Friend Module BusinessOperationExceptionExtensions

    <Extension()> _

    Friend Function DBExceptionCode(ByVal ex As BusinessOperationException) As String

        Return ex.DataFormErrorInfo.SQLErrorInfo.ErrorMessage

    End Function

End Module

Now for any BusinessOperationException ex, you can just call ex.DbExceptionCode instead of traversing the various properties. Sweet!

Lambda Expressions

According to MSDN, a lambda expression is "a function without a name that calculates and returns a single value." Think of a lambda expression as an executable block of code that is defined in-line, rather than in a function that must be defined somewhere else. A lambda expression can also make use of variables that are in scope at the point where it is defined. Because a lambda expression helps you keep the flow of logic in one place and avoid the ceremony of passing parameters, it can serve as a powerful tool for simplifying your code.

You will recall that in our sample code, the .Where extension method processes a lambda expression:

.Where(Function(e) e.DBExceptionCode.Contains("ERR_MEMBERSPECIFIEDMEMBERS_INVALID"))

The Function keyword indicates that the expression which follows is a lambda. Behind the scenes, the VB compiler will do the following:

build an anonymous function that takes a parameter of type BusinessOperationException and returns Boolean; the lambda expression then becomes the body of the function.
infer that the generic type TException declared in the .Where() method definition will be of type BusinessOperationException in this block of code.
build a delegate to that anonymous function, and then pass that delegate as an argument to the .Where extension method, which has a ByVal predicate As Func(Of TException, Boolean) parameter.

And you thought you worked hard! Well, why not let the compiler do the hard work for you? You could think of a lambda expression as a way to get the compiler to handle all the plumbing chores of function bodies and delegates, while leaving just the essence of the logic visible in your code.

Conclusion

Extension methods and lambda expressions can help you write code that is more succinct, more readable, and more powerful. According to Microsoft's Eric White, "Functional Programming (FP) has the potential to reduce program line count by 20% to 50%, reduce bugs and increase robustness, and move us in the direction of taking advantage of multiple core CPUs." In this article we saw how extension methods and lambda expressions, which are important parts of the VB functional programming toolkit, made the sample test code more readable and succinct. Of course these new techniques are not a golden hammer that can be used everywhere; nevertheless, be alert for opportunities to use them in the test code you write.

Source Listing for AssertExtensions.vb

Imports System.Diagnostics

Imports System.Runtime.CompilerServices

Imports System.Text

Public Module AssertExtensions

    ''' <summary>

    ''' Verifies that a specific method call throws an exception of a specific type

    ''' </summary>

    ''' <typeparam name="TException">The type of exception the method call is expected to throw</typeparam>

    ''' <param name="workToDo">delegate to method call which is expected to throw an exception</param>

    ''' <returns>

    ''' instance of exception thrown by the Action

    ''' </returns>

    ''' <remarks>

    ''' This method will re-throw any exceptions of a type other than that designated by the generic TException.

    ''' If the Action does not throw any exceptions, this method will throw an AssertFailedException.

    ''' </remarks>

    <Extension()> _

    Public Function ShouldThrow(Of TException As Exception)(ByVal workToDo As Action) As TException

        Dim errorMessage As New StringBuilder("An exception of type ")

        errorMessage.Append(GetType(TException))

        errorMessage.Append(" should have been thrown, but wasn't.  Test method info follows...")

        errorMessage.AppendLine()

        errorMessage.AppendCallingMethodInfo()

        Return ShouldThrow(Of TException)(workToDo, errorMessage.ToString())

    End Function

    ''' <summary>

    ''' Verifies that a specific method call throws an exception of a specific type

    ''' </summary>

    ''' <typeparam name="TException">The type of exception the method call is expected to throw</typeparam>

    ''' <param name="workToDo">delegate to method call which is expected to throw an exception</param>

    ''' <param name="errorMessage">user-defined error message if expected exception not thrown</param>

    ''' <returns>

    ''' instance of exception thrown by the Action

    ''' </returns>

    ''' <remarks>

    ''' This method will re-throw any exceptions of a type other than that designated by the generic TException.

    ''' If the Action does not throw any exceptions, this method will throw an AssertFailedException.

    ''' </remarks>

    <Extension()> _

    Public Function ShouldThrow(Of TException As Exception)(ByVal workToDo As Action, ByVal errorMessage As String) As TException

Try

            workToDo.Invoke()

            Throw New AssertFailedException(errorMessage)

        Catch ex As Exception

            If (GetType(TException).Equals(ex.GetType)) Then

                Return ex

            End If

            Throw

        End Try

    End Function

    <Extension()> _

    Public Sub AppendCallingMethodInfo(ByVal sb As StringBuilder, Optional ByVal howManyFramesBack As Integer = 1)

        ' how far back is from the perspective of the caller of this method. +1 gets us back to the frame of

        ' this method's caller, then we continue from there

        howManyFramesBack += 1

        Dim st As New StackTrace(fNeedFileInfo:=True)

        Dim sf = st.GetFrame(howManyFramesBack)

        sb.Append("Test method: ")

        sb.Append(sf.GetMethod())

        sb.AppendLine()

        sb.Append("FileName: ")

        sb.Append(sf.GetFileName())

        sb.AppendLine()

        sb.Append("Line Number: ")

        sb.Append(sf.GetFileLineNumber())

    End Sub

    ''' <summary>

    ''' Verifies that an exception satisfies a predicate

    ''' </summary>

    ''' <typeparam name="TException">The type of exception against which the predicate is applied</typeparam>

    ''' <param name="ex">The exception against which the predicate is applied (passed by compiler)</param>

    ''' <param name="predicate">The condition which the exception must satisfy</param>

    ''' <returns>True if exception satisfies predicate, otherwise an AssertFailedException is thrown</returns>

    ''' <remarks>

    ''' Unlike the IEnumerable(Of T).Where extension method, this method does not return a subset from a set.

    ''' Instead it verifies that a particular instance of an exception satisfies a predicate.  This method is

    ''' designed to be used in conjunction with ShouldThrow to provide a fluent expression for a test.

    ''' <example>

    ''' Dim myAction As Action = (Function() MyMethod(myData))

    ''' myAction.ShouldThrow(Of SpecializedException)().Where(Function(ex) ex.SpecialProperty = "Hello")

    ''' </example>

    ''' </remarks>

    <Extension()> _

    Public Function Where(Of TException As Exception) _

    (ByVal ex As TException, ByVal predicate As Func(Of TException, Boolean)) _

    As Boolean

        Dim errorMessage As New StringBuilder("The " + GetType(TException).ToString() + " did not satisfy the Where predicate.  Test method info follows...")

        errorMessage.AppendLine()

        errorMessage.AppendCallingMethodInfo()

        Return Where(ex, predicate, errorMessage.ToString())

    End Function

    ''' <summary>

    ''' Verifies that an exception satisfies a predicate, and throws exception with user-supplied message if not

    ''' </summary>

    ''' <typeparam name="TException">The type of exception against which the predicate is applied</typeparam>

    ''' <param name="ex">The exception against which the predicate is applied (passed by compiler)</param>

    ''' <param name="predicate">The condition which the exception must satisfy</param>

    ''' <param name="errorMessage">Message of AssertFailedException thrown if predicate fails</param>

    ''' <returns>True if exception satisfies predicate, otherwise an AssertFailedException is thrown</returns>

    ''' <remarks>

    ''' Unlike the IEnumerable(Of T).Where extension method, this method does not return a subset from a set.

    ''' Instead it verifies that a particular instance of an exception satisfies a predicate.  This method is

    ''' designed to be used in conjunction with ShouldThrow to provide a fluent expression for a test.

    ''' <example>

    ''' Dim myAction As Action = (Function() MyMethod(myData))

    ''' myAction.ShouldThrow(Of MyException)().Where(Function(ex) ex.MyProp = "Hello", "MyProp wasn't 'Hello')

    ''' </example>

    ''' </remarks>

    <Extension()> _

    Public Function Where(Of TException As Exception) _

    (ByVal ex As TException, ByVal predicate As Func(Of TException, Boolean), ByVal errorMessage As String) _

    As Boolean

        If Predicate(ex) Then

            Return True

        Else

            Throw New AssertFailedException(errorMessage)

        End If

    End Function

End Module

How to Reuse Code Without Creating an Implicit API

Chris Falter — Sun, 29 Mar 2009 19:44:23 GMT

I work for an ISV, and we have to be very cautious about the code that we declare with public scope. If it's public, a customer can treat it as an API, which might not always be a good idea if the code is really just doing some internal task in a way that could easily change as we improve our software. In other words, not all that is public should be an API.

So what do you do if you need to share code across assembly boundaries? Recently I was writing some unit tests for a class, and the tests needed access to a class method in order to imitate (and test) its behavior. Of course this method was private--of course!--so it appeared that I had to choose between 2 very unpalatable options:

Give the method public scope, which would make it visible to all our customers' implementations. Essentially, this option would make the method part of our API. Not happening.
Copy the 20 lines of code from the method, paste it into my new class, and supplicate the dark spirits of bad software design to supernaturally prevent the method from ever being modified. But how could I perform the ritual without an eye of newt and toe of frog?*

But appearances can be deceiving. It turns out that the InternalsVisibleToAttribute, introduced by Microsoft in .NET 2.0, solves the problem quite neatly. Using this attribute, you can designate one or more friend assemblies that will be allowed access to classes, methods and properties in the target assembly that have friend scope, while all other code--including customer code--is denied access.

Using the attribute is a piece of cake:

Mark any classes/methods/properties that need to be visible to other assemblies with "Friend" scope.
Import the System.Runtime.CompilerServices namespace into your assembly's AssemblyInfo.vb file.

Imports System.Runtime.InteropServices
In AssemblyInfo.vb, mark the assembly with the InternalsVisibleToAttribute for each friend assembly that should have access to the friend scope.

<Assembly: InternalsVisibleTo("MySystem.Assembly1")>
<Assembly: InternalsVisibleTo("MySystem.Assembly2")>

If you are working in C#, you will make your changes in the AssemblyInfo.cs file for the assembly that contains the code to be re-used, and you will use "using" instead of "Imports."

So I could have my cake (re-use of existing code) and shield it from inappropriate eyes at the same time. While this technique is obviously useful for writing tests, it can be applied anywhere that code needs to be shared between your system's assemblies without exposing it customers as part of an API.

* Not to mention the wool of bat and tongue of dog.

Why You Should Consider Using FIT

Chris Falter — Tue, 05 Feb 2008 22:12:48 GMT

I was looking for ways to improve our development process when I stumbled across the Framework for Integrated Testing (FIT). The rest of this post is the body of an email I recently sent to my manager.

The goal of the Customer update is to migrate all their customizations to a new framework in a reliable and quick manner. The major risks I have been able to identify are as follows:

Developers won’t be able to identify all the behaviors that need to migrate.
- Requirements are scattered.
- Code is not organized optimally. Much business and workflow logic is scattered throughout the UI layer, in particular. Many customizations in domain logic are located in poor locations in the domain class hierarchy; the root domain object seems to have become a catch-all location for just about anything.
Manual regression testing will become a bottleneck. It is already a serious bottleneck for other projects.
Poor code organization will be perpetuated into the new source code base. As a result, the project will proceed more slowly (due to high bug rate and unnecessary complexity), and subsequent evolution of the code will be hampered as well.
Lack of an automated testing framework will impede any efforts to refactor code. Since this project by definition requires an extensive reworking of our source code, this risk is quite serious.

Note to readers: I'm sure many of you have faced these kinds of challenges on one or more projects. Read on to discover a framework that can help you can meet the challenges!

While unit testing would go a long way toward addressing the third and fourth risks, it would not help us address the first 2 risks. We can ask business analysts (BAs) to help developers identify migration requirements, but we don’t have a repository for storing these requirements. Even if we implement such a repository, a developer must still translate requirements (in whatever form they exist) into unit tests, which introduces the possibility of error. Finally, the unit tests are not visible to the BAs, so they cannot rely on them for regression testing.

The Framework for Integrated Testing (FIT) seems to address all four of these risks quite adeptly. Before I proceed, I should explain that FIT allows a non-developer to write requirements in tabular form. FIT provides 3 types of test fixtures that can be used in a table:

A ColumnFixture is useful for business rules, as it is used to declare a set of inputs and expected output(s). For example, you could use a column fixture to declare that a dwelling in a particular location built in 1996 should have a BCEGS of 99, but if it is built in 2004 it should have a BCEGS of 4. As the Customer systems are replete with custom rules (such as underwriter referral rules), the ColumnFixture should prove very helpful.
A RowFixture is ensures that data are being stored and retrieved correctly from a data repository. While I see no need to test the data layer’s reliability with respect to standard data such as dwelling address (it has proven quite reliable for other projects), the RowFixture could help us with data that are unique to Customer (such as subdivision).
An ActionFixture is used to simulate workflow requirements. For example, a tabular ActionFixture could state that when a user enters a dwelling limit of $10k and presses “Continue”, a certain error message should appear; when s/he enters a dwelling limit of 300k and presses “Continue”, the system advances to the Quote Summary page; and if s/he enters a dwelling limit of 1100k, the system advances to the Qualification Questions page (due to consent to rate rules).

If we use the WinFitRunnerLite tool, a BA can construct the test data tables in Excel, which may be more intuitive than writing them in Word and saving as HTML.

A developer must write a test fixture in order to translate the test data into an actual test. The test fixture will link to the domain, data, and controller/workflow libraries of the system under test, pass data from the test document’s tables to the appropriate methods, and evaluate the results against the BA-supplied expectations. If we follow this approach, it will be impossible for a developer to embed workflow logic, rules, or domain logic in the UI (a big win!).

One of the other advantages of FIT is that a BA can “own” the generation of requirements and tests, since they are maintained in a single location. A BA can easily design the test scenarios that the system must pass; I’ve seen complex requirements in Excel spreadsheets plenty of times, so I have no doubt about the ability of our BAs to generate FIT scenarios. On the other hand, how many BAs could write a unit test?

Using FIT can help answer many project management questions:

Developer: How have requirements for the system changed? Answer: check the FIT document.
Developer: What are the requirements I should work on? Answer: see which FIT tests have not yet passed.
BA: How close are we to completing the latest project milestone? Answer: run the FIT tests and see how many tests have not yet passed.
BA: Do the latest changes contain any regressions? Answer: run the FIT tests and see if any tests that used to pass have started to fail.

It is true that the FIT tests do not run against the actual UI. I see this is a positive, as it will prevent the project team from falling into the “Smart UI” trap. However, it also means that FIT, however useful I think it will be, is not a silver bullet; testers will still have to interact with the system’s UI.

Nevertheless, I think FIT will prove very helpful for this project, and for our shop in general. And since BAs have to write requirements and perform testing, and developers have to translate requirements into testable code anyway, using FIT should not be burdensome.

A Tool for Troubleshooting Web Apps and Web Services

Chris Falter — Mon, 28 Jan 2008 13:20:45 GMT

Recently I discovered a tool that I had needed for a long time. Fiddler, a freeware product created and supported by Microsoftie Eric Lawson, can be used to debug HTTP traffic from any web browser (or client application). Here's how you can use it:

1. View/Analyze HTTP traffic.

One of our web apps creates a new window and performs a javascript-driven auto-logon to a vendor application. When we reported an issue to the vendor, they started asking questions about whether we were using the correct version of their javascript module. Rather than attempting to debug the javascript, I simply used Fiddler to capture all the HTTP traffic between my browser and the vendor's web server. Fiddler formatted the data quite nicely, which made verifying my browser's behavior quite simple. Fiddler was also able to save the relevant portions of the HTTP conversation to a log file, which I forwarded to the vendor. The vendor was able to determine quickly that the error was in the behavior of their web server, rather than in any incorrect use of javascript on our part.

Just one day later, another vendor questioned whether one of our web services was correctly returning error data. Source code-level debugging in the web service would have done me no good, since I had to verify the data being transmitted on the wire. But Fiddler made the investigation easy. I used Fiddler to capture an interaction between a test client and the service, and quickly verified that our service was returning the correct error data.

You can also use Fiddler to view all the details of a secure HTTP interaction. Try doing that with netmon! Just configure Fiddler to view HTTPS traffic, and it will hook any secure HTTP transactions. It uses the "man-in-the-middle" strategy: it sets up a secure HTTP transaction between the browser and itself, then transmits the data it receives to the target URL over secure HTTP. Sitting in the middle, it is able to view all the data in the browser-server conversation. You just have to tell your browser to ignore the any certificate errors associated with the temporary certificate that Fiddler generates.

2. Generate test scripts.

The Visual Studio 2005 Test Edition has a tool that allow you to generate a test script for a web app by capturing the interaction between a browser and a web app. However, this tool only captures interactions (such as form submits) generated by clicking on links and buttons. So how do you capture javascript-driven AJAX interactions? Use Fiddler!

3. Analyze performance enhancements.

If you are trying to figure out how various browser settings will affect a website's performance, Fiddler can help. You can script Fiddler to intercept browser interactions and alter metadata, which allows you to perform a "what if" analysis on the fly. And Fiddler has a built-in timer to measure wall clock time during the interaction.

The Fiddler site has extensive documentation, including some good QuickStart videos. If you've been looking for a good tool to analyze HTTP traffic, look no more; go directly to the Fiddler download page and get started!

Test-Driven Design: Make Sure You Fail!

Chris Falter — Sat, 20 Oct 2007 18:39:53 GMT

The idea behind test-driven design (TDD) is to incorporate testing into the process of constructing your system, rather than waiting for developers to check in their code and to build a system that someone else tests. TDD has some advantages worth considering:

Defects are identified sooner in the development process, which makes delivery of a reliable system cheaper and quicker.
Developers have more scope to improve the quality of code by refactoring, since they can run unit tests as they are making changes to verify whether the system still behaves as expected.
Unit tests serve as built-in documentation on how to call and make use of interfaces.
This built-in documentation is guaranteed not to go out of date. (You've never had the joy of working with out-of-date documentation, have you? You haven't lived until you've debugged some code whose documentation does not match. Or perhaps I should say, you haven't worked in software development unless you've debugged, etc.)
A team can run unit tests in the final step of the build process, in order to verify that the new build has no obvious defects.

The test-driven design process goes like this:

Get the requirement from the customer.
Design a test suite that will verify that the requirement is satisfied.
Run the test suite and make sure it fails.
Write code and run the test suite until it succeeds.

When the test suite succeeds and there are no obvious ways to simplify the code, the developer performs his customary victory dance and checks in the code.

I have incorporated TDD into the development process for some web services that have multiple customers. If we did not have a more or less complete suite of unit tests, any modification would be hard to manage. Any codepath that any customer relies on could have a regression when any code is changed, right? So in the absence of TDD, any modification would require that several testers (Pat, Tammy, and Charles at our office) spend a fair amount of their precious time running regression tests. Consequently, I have created suites of unit tests to verify that no regressions have occurred, and have even given them names ("virtual Pat," "virtual Tammy," and "virtual Charles") . Virtual Pat and her virtual colleagues are working hard so that the real Pat and her colleagues can head home at 5pm.

Recently I received a requirement to modify a service method so that it would return extra data. After I revised the unit tests, I almost skipped over step #3 (make sure the test fails). After all, I was in a hurry, and why bother? Aren't my coding skills good enough to write a simple test?

Evidently not.

I ran the test suite and it did not fail. I rubbed my eyes in disbelief and ran the suite again; of course, it succeeded again. A quick glance at the tests showed that the test suite contained the extra data expected from the method, but I had failed to write the assertions that would compare the expected data with the actual data returned from the method. So if the actual data returned from the method was not correct, the test (and the developer) would never know it. Doh! <Sound of hand smacking forehead />

So I added the assertions, ran the test suite (Hurrah! It failed!), wrote the new code, ran the test suite again, and it succeeded. I checked in my code, and it's working as expected.

The moral of this tale is simple: in test-driven design, you have to fail before you can succeed.

How To: Write a Unit Test for Multi-Dimensioned Output

Chris Falter — Fri, 04 May 2007 13:42:42 GMT

One of the cool features of Visual Studio 2005 is that it can generate a unit test stub for a class or method. However, I recently discovered that the Unit Test plug-in does not know how to auto-generate an assertion for a method whose output includes a multi-dimensioned out parameter or return value. Here's the relevant stub code that VS 2005 generated for one of my methods whose output includes a multi-dimensioned array:

// generated code that creates a class instance called "target", plus other input + expected output parameters

string[][] homeAttributes;
string[][] homeAttributes_expected = null; // TODO: Initialize to an appropriate value

target.MethodBeingTested(in1, in2, out homeAttributes)

CollectionAssert.AreEqual(homeAttributes_expected[i], homeAttributes[i], "homeAttributes was not set correctly.");

So I edited the code to initialize homeAttributes_expected, then ran the unit test, and it failed. Okay, my code must have a bug...but when I reviewed the code for MethodBeingTested(), it sure looked right. Then I stepped through the unit test in debug mode. However, when I hit the breakpoint on CollectionAssert.AreEqual() and expanded the arrays, the data was exactly what I expected it to be. So maybe my initialization code for homeAttributes_expected was wrong...but no, it looked right. What was wrong with this picture?

(These are the kind of challenges that either bring out the best or the worst in a software developer. You either rise to the occasion, or become a broken, miserable, and somewhat sneaky creature that silently excises the CollectionAssert.AreEqual() statement so the unit test will pass. And I was not about to steal away quietly!)

I realized what must be happening: CollectionAssert.AreEqual() must be calling the object.Equals() method to compare each member of the collections. And in a multi-dimensioned array, the collection object being tested (the array of arrays) consists of members that are themselves collections (i.e., arrays). Since the built-in Array class in the .NET Framework does not override the object.Equals() method, CollectionAssert.AreEqual() will use the default object.Equals() method to compare the identity of the array members; in other words, it compares the address of the 2 arrays in memory. These will of course be different, so the assertion fails, and the unit test fails.

You have probably noticed that this post claims to apply to all multi-dimensioned output. That's because you will encounter the same problem if your output is a List<string>[], or a List<List<string>>, or any other multi-dimensioned object.

Fortunately, you can conquer this problem with some clever refactoring. Just iterate through the outer dimension, and call CollectionAssert.AreEqual on the inner collections. Write your code like this, and everything will work just the way you want it to:

for (int i = 0; i < homeAttributes_expected.GetLength(0); i++)
{
CollectionAssert.AreEqual(homeAttributes_expected[i], homeAttributes[i], "MethodBeingTested_homeAttributes_expected was not set correctly at index " + i.ToString());
}

Now CollectionAssert.AreEqual() compares two single-dimensioned collections each time it is called. As a result, when it calls object.Equals() on the members of the collections, it will use the member type's override of object.Equals()--in this case, String.Equals()--and the assertion works correctly.

Happy unit testing!