Software Development Practices

Are You Mocking My Code?

Russell Ball — Wed, 19 Dec 2007 06:12:44 GMT

I've been playing around with mock objects using Rhino.Mocks for the last few months, but today was the first time that I had an unfettered coding win and became a true convert.

I was introduced to NUnit about five years ago from a co-worker (thanks Dewayne) and gradually became a true believer. Unfortunately, I was a late-bloomer in the area of mocking, so by the time I finished my last big project I had left a suite of about 500 database driven tests that took almost 2 hours to run and were a maintenance nightmare. The vast majority of my unit testing efforts revolved around SQL initialization and cleanup scripts that massaged the database into just the state I needed for each tests case scenario.

Although this approach definitely made me a wizard when it came debugging data-related production issue quickly, it was too high of price to pay for the benefits I was receiving. Although I remember thinking in the beginning that this "realistic" approach to testing was much more valuable than the "fake" tests that didn't hit the database, by the end of the process I was no longer so sure. I still believed in the value of unit testing, but I knew there had to be a better way of doing it.

During my stint as an architect, I read enough about mocking to know that it was a good idea in theory, but of course I had little opportunity to put that theory to the test. Now that I am a code monkey again, I can say with certainty that mocking is an absolutely essential aspect of Test Driven Development and not simply a "nice-to have" supplemental activity like I assumed before.

Why is Mocking Critical?

It makes debugging easier - When a database driven test fails, the problem could be anywhere and in my experience it is most often related to a problem with the initialization script (i.e. accidental order-based test dependencies). This often led me to ignore test failures when I was pressed for time because I knew that it was likely just another false alarm. However, when a test that uses mock objects fails, I am not only fairly certain that it is a problem with the code rather than the test, but I also know exactly where the problem is because I've guaranteed the behavior of every piece of code except the part that I am trying to test.
It makes writing tests faster and easier - One of the biggest barriers to adoption for TDD for most developers continues to be the perception that it takes too long. I can give a dozen reasons why that perception is wrong, but in the end I think it is more productive to simply find ways to shorten that process. That is exactly what mocking does. Writing out a series of Expect.Call() statements is an order of magnitude faster than trying to write data manipulation statements.
It makes tests execute faster - Don't underestimate the power of fast feedback. Even if you are not a die-hard continuous integration practitioner, you have to admit that the longer your tests take to run the less likely you and other developers will be to actually run them. Even if you do religiously run them at night, the longer feedback cycle dramatically decreases one of the main benefits of TDD, which is the increased coding speed that comes from test-backed confidence and not having to waste time on excessive analysis and fretting about possible unknown collateral damaged.

What does Mocking Require?

Interface Based Programming - If you use Rhino.Mocks, then every class that you mock must implement an interface. With the help of ReSharper, interfaces can be created from an existing class and propagated to the rest of your codebase with just a few keystrokes. This alone makes the price of ReSharper worth it if you are going to start using mocking in a Legacy system that was designed without using interfaces.
Dependency Injection - One of the first stumbling blocks I ran into came from complex collaborations from objects that weren't able to be mocked because they were created inside of methods. You can quickly get around this with a little refactoring by making collaborating objects private member variables and then adding a constructor overload to expose them during testing. It is usually a relatively fast and low-risk design change to make.
Identifying Code Seams - If you find yourself getting frustrated and beginning to think that you are stuck in a code base that is impossible to mock, then stop and read the book Working Effectively With Legacy Code by Michael Feathers. I'm about a quarter of the way through this right now and it is one of the most helpful tech books I've ever read. The author is fearless and infinitely resourceful when it comes to getting code of any language and any quality under test. In fact, many of his examples are in C++ and even C, so that means that you have no excuses when it comes to your own project. He gives lots of practical advice on how to find the "seams", which are the easiest and most cost-effective places to modify code so that you can begin to mock it.

A Practical Example?

Here is an MbUnit test that I wrote today before I tackled a production bug that was assigned to me. It mocks the IPaymentTypeItem, which has read-only properties and thus is normally only allowed to be populated from the database. The Expect.Calls() are just done for the methods and properties calls that are actually made on the object in the piece of code I am testing. You can get a full tutorial with better explanations of the API calls here.

   1: [RowTest]

   2: [Row("%B409999999999999^Russ's Coffee Emporium^09011211000019900000000?;","409999999999999")]

   3: [Row(";4409999999999999=090112110000199?","409999999999999")]

   4: public void Should_Load_Correctly_With_Partial_Track(string trackData, string expCardNumber)

   5: {

   6:     MockRepository mocks = new MockRepository();

   7:     IPaymentTypeItem payment = (IPaymentTypeItem) mocks.DynamicMock(typeof (IPaymentTypeItem));

   8:     Expect.Call(payment.BinFirstDigit).Return(expCardNumber.substring(0,1));

   9:     Expect.Call(payment.IsCreditCard).Return(true);

  10:     Expect.Call(payment.PaymentTypeValue).Return(PaymentType.VisaCard);

  11:     mocks.ReplayAll();

  12:     FinancialCardEncoding encoding = FinancialCardEncoding.CreateCardEncoding(trackData,payment,true);

  13:     Assert.AreEqual(0,encoding.Errors.Count);

  14:     Assert.AreEqual(expCardNumber,encoding.PrimaryAccountNumber);

  15:     mocks.VerifyAll();

Rhino.Mocks is certainly not the only mocking framework out there, but so far I like it because it takes a strongly typed approach rather than relying on strings which means I get to use intellisense. Whatever framework you do choose, I highly recommend that you take the time to learn it well and incorporate it thoroughly into your development as soon as possible.

Still an Open Source Virgin

Russell Ball — Tue, 06 Nov 2007 14:10:59 GMT

I was setting up Cruise Control the other day and trying to figure out why it wasn't working on a certain source control folder. I noticed a strange error in the cruise control log about there being an invalid character in the path, so I decided to take advantage of the fact that it is an open source project and download the source code so I could step through it in the debugger and see what the exact problem was.

It wasn't long before I discovered the source of the error, which was a newline character embedded in the source control folder name. Since I am a vocal Visual Source Safe hater, I naturally assumed that the problem had to do with VSS data being corrupted so I scheduled some VSS maintenance and called it a day.

When VSS analyzer did not fix the problem, I took a closer look and realized that I had unfairly blamed VSS and the real problem was a bug in the Cruise Control code that parses out the output from the VSS command line tool (thank god PowerShell eliminates the need for much of this parsing voodoo). If the folder path in VSS is too long, then the command output wraps and Cruise Control incorrectly inserts a newline character inside the folder path.

The problem was easy enough to fix by stripping out the rogue newline character, so I recompiled Cruise Control, replaced the problem dll, and all was well in the world.

At that point, it occurred to me that other people must be running into this problem, so I decided to try being a good open source citizen for once and submit the bug fix. I had never tried this before, so I read the contribution procedures posted on the home page and followed their polite suggestions. I created a diff file of my fix, wrote up a detailed description of the problem, and even found a unit test that someone had commented out that failed under the current code base and worked with my fix.

I was just about ready to submit a JIRA ticket in their bug tracking system, when it occurred to me that I should probably browse the current VSS tickets first. It turns out that my issue was not only recorded in the system, but had just been marked as resolved by someone else the week before. Doh! I guess I should have tried that first, huh?

Oh well, at least I got practice going through the procedure for being an open source contributor. There are still an lots of open tickets for the current Cruise Control release, so perhaps I'll grab one of them while I have everything already set up.

After all, I don't want to be an open source virgin forever.

What are your Seven Essential Practices?

Russell Ball — Sat, 20 Oct 2007 04:07:44 GMT

Ayende started a thread in the ALT.NET newsgroup asking participants what practices they deemed absolutely necessary on a project. In other words, if you were to start in a shop that had none of the current "best practices" in place, which ones would you spend your political capital on trying to get into place first.

If I only had enough time, money, and political capital to accomplish seven things, I would choose the following:

Test Driven Development: This is the one practice that I can't really live without. I can be patient if others decide not to use it (although I will quietly try to win them over), but I would be in a real moral quandary if I wasn't allowed to practice this myself for some reason. It enables me to capture requirements concretely, improve my design by "dog-fooding" my own API, and refactor without fear because I have at least some assurance that I haven't broken anything.
Source Control: I don't really consider VSS a legitimate option, but as long as I don't have to do much branching I can live with it in the short term (as I am doing now). At least VSS allows you to easily identify changes in a file and gives you a fair shot at rolling back individual files. No source control is simply not an option.
Continuous Integration: The more people you have, the more necessary this becomes. The longer the feedback loop, the more pain you incur by fixing integration problems. At minimum, you need to have a controller (i.e. CruiseControl.NET) monitoring your source control system and automatically building your projects and running your unit tests (at least the fast running ones) upon check-in. Longer running tests can be scheduled hourly or nightly. Visibility (i.e. stoplight) is key if you want your team to get the most out of this practice.
One Step Deployment: There is nothing more frustrating than having to take responsibility for deployment related errors that occurred in your application because a manual step was accidentally omitted. You may know that it was the deployment process that sucks, but everyone else will assume it is your code.
Bug/Feature Tracking: The place I work at now uses JIRA, which is probably the best tool I've used so far in this category. If you can't effectively track your work, then you have no chance of getting it done right on a consistent basis.
Developer Productivity Tools (Resharper/CodeRush): I bought my own personal license for Resharper so I wouldn't have be forced to go without. The more keyboard shortcuts I have at my disposal, the faster I code and the better chance I have to get and stay in the zone where I am focused just on solving the problem at hand rather than getting distracted by syntax and unnecessary keystrokes.
Iterative SDLC: I simply refuse to work on another death-march, multi-year waterfall project. Although I think mature agile shops are by far the best working environments, I can be happy in more traditional shops as long as they follow some basic principals in spirit. Releases must be frequent and there has to be some interactive way to capture user requirements. Work should be continually re-prioritized based both on business value and according to a realistic budget based on average team velocity. Most developer overtime I witness seems to be avoidable and a direct result of the break-down in one of these fundamental practices.

If you only had seven to choose from, is there anything that you would bump off this list in favor of a practice you find more essential?

In Praise of MbUnit

Russell Ball — Fri, 19 Oct 2007 22:22:34 GMT

I've played with MbUnit before and was immediately impressed, but I haven't tried using it on a project until today. I was surprised by how quickly and easily I was able to convert all of my NUnit tests. All I had to do was switch the references from NUnit.Framework to MbUnit.Framework and then do a quick find and replace for my using statements. After that everything magically worked. Kudos to the MbUnit team for not pulling an 'MSTest' move and renaming all the attributes and keywords.

Best of all, I was able to add several new tests quickly by simply adding parameters to my test methods and then using the MbUnit Row attribute to dynamically pass in multiple values. This simple feature not only helped eliminate quite a bit of duplicate test code, but it also encouraged me to exercise my code with more test data which immediately led me to find some new bugs.

I especially like how the MbUnit test runner lists each parameter as a separate test so you can easily see which one of the data elements caused the problem. It's also nice to have multiple views of the test at the same time (Author/Category/Namespace).

If you haven't already tried using MbUnit, I highly recommend it. There is almost no effort involved in learning to use it, especially if you've ever used NUnit.

Taking Another Look at BDD

Russell Ball — Wed, 03 Oct 2007 06:03:10 GMT

When I first heard about Behavior Driven Development, I dismissed it as a trivial re-branding of TDD (Test-Driven Development). I figured that either someone was getting way too anal about "getting the words right" or else a concerted effort was being made to make TDD more palatable to the skeptics in the same way that Agile made Extreme Programming principles more mainstream through the magic of less controversial language. It's hard not to be cynical about re-branding efforts when you've had to endure years of corporate culture propaganda and a lifetime of marketing voodoo.

Since BDD is one of the topics that will be discussed at the ALT.NET conference that I will be attending this weekend, I decided to challenge my initial assumption and do some verigoogling on the topic. I am pleased to report that I have changed my mind about BDD and now believe that there is enough significant content in this mini-methodology that is beyond the realm of traditional TDD to merit some serious discussion.

If you want a quick introduction into how BDD is different from TDD, I recommend reading the article Introducing BDD by Dan North, who appears to have coined the phrase. He starts by relating the common problems he had as a TDD coach and then describes different evolutions in his thought process and how a fundamental shift in his orientation ultimately helped alleviate these problems. He concluded that the word "test" was partially responsible for students overlooking the critical design-oriented benefits of writing tests first and that changing the focus from an application's state to its behavior naturally led practitioners to more sophisticated and effective TDD practices.

One concrete example of how this different perspective alters the way you write tests has to do with naming. One BDD practice is to name tests using simple sentences that start with Should (i.e. ShouldFailForMissingSurname) and use Ubiquitous Language that your business users would easily understand when choosing the words. This is one of the areas where Domain Driven Design, one of the three pillars of BDD, comes into the mix. Besides allowing tools like NSpec to actually produce usable documenation from your tests, this naturally forces developers to limit the scope of their tests (you can only fit so much into a name), which in turn leads to better design and less debugging with the IDE debugger.

If you want to learn more about this practice, I would first suggest perusing Behaviour-driven.org to get a higher level overview and then digging into the blog posts under Scott Belware's BDD Category to get some deeper insight into these practices. Jeremy Miller also has a good post entitled BDD, DDD, and other Double D's.

Analyzing WatiN Code Quality with NDepend's CQL

Russell Ball — Tue, 18 Sep 2007 13:06:54 GMT

In my recent post on code quality, I mentioned some of the high level features of a static analysis tool called NDepend. I've been exploring some of the pre-built CQL (Code Query Language) queries in the tool by running them against the WatiN codebase. Here are the results from selecting "Types with Too Many Methods" query. NDepend shows you a list of the classes and number of methods in each on the left along with a visual Tree Map visual representation on the bottom.

Once code is analyzed while first loading the assembly, the GUI is blazingly fast and dynamically updates whenever you highlight a new query. You can modify queries to suite your own standards and then double click to drill into the exact spot in the code. Here is what the CQL query looks like for identifying types with excessive methods:

WARN IF Count > 0 IN SELECT TOP 10 TYPES WHERE NbMethods > 20 ORDER BY NbMethods DESC

Here are some other pre-built queries that I found especially compelling:

Methods too big ( > 30 lines of code)
Methods with too many parameters ( > 5)
Too many local variables ( > 15)
Methods too complex (cyclomatic complexity > 20)
Poorly commented (% comment < 20 and # lines > 10)
Fragile base class (depth of inheritance > 6)

NDepend showed that WatiN had the following issues that could potentially be refactoring opportunities.

One method with too many parameters (SendMessageTimeout has 7)
Ten classes with too many methods (Document has 97)

There are also a very helpful group of queries that are designed to just help you better understand the codebase by identifying the most used methods, types, and namespaces.

As an interesting aside, this tool seems to have been written entirely by one person, Patrick Smaccia. I want to personally thank him for giving me indisputable proof that I barely qualify for a bottom-dwelling, plankton-like status on the alpha-geek food chain.

Code Quality: The Holy Grail of Software

Russell Ball — Mon, 17 Sep 2007 06:16:59 GMT

Monitoring and enforcing code quality seems to be somewhat of a holy grail in the software industry in that nearly every development shop pursues this goal but few ever even come close to actually achieving it. Here are a few of the common failed approaches I've seen:

Developer's Handbook - Despite being a darling of auditors, I find this approach to be a largely worthless exercise for anyone except perhaps the author of the document. Like all waterfall-based functional specifications, these documents quickly become outdated and are seldom read and often misinterpreted by developers. Even in ideal circumstances where developers are highly motivated to follow guidelines that are well understood, this honor system approach falls hopelessly short simply because of basic human fallibility. Logical mistakes in code are caught because users, testers, or unit tests complain about something not working, but how are code quality mistakes caught?
Code Reviews - After becoming disillusioned with the honor-based rules approach, many shops attempt to supplement their developer handbooks with a manual code review process. Although a second set of eyes increases the odds of catching errors, this approach is extremely expensive to implement and subject to all sorts of emotional pitfalls that can easily turn the review into either a meaningless rubber-stamping exercise or an ugly battle of egos. Even with the most mature and disciplined developers, this approach ultimately fails to provide the level of code quality that shops are seeking due to a lack of knowledge or simple oversights by the reviewers.
Architects - In order to maximize the effectiveness of reviewers, shops might take the step of designating a specialist, such as an architect, to be in charge of overall code quality. Unfortunately, whatever advantages that this position brings to the review process in terms of technical knowledge and increased authority are often nullified by the specialist being too far removed from the code to appropriately identify issues or too far removed from the team to effectively enforce changes. Even when the specialist is viewed with respect rather than ridiculed as an out-of-touch Architecture Astronaut, suggestions tend to be seen as too subjective to be actionable when compared to hard deadlines.

Now for some more promising options that I've seen...

FxCop - This tool analyzes source code and produces a list of violations where code doesn't comply with a set of preconfigured rules, such as naming standards. By relying on software rather than people to enforce code quality rules, you dramatically decrease the chance for error, the perceived subjectiveness of the process, and the overall cost. Unfortunately, static analysis tools such as FxCop are only capable of monitoring a certain class of the more superficial rules and don't catch logical errors or help identify more complex code quality issues such as cyclomatic complexity, low cohesion, excessive dependencies, or overly complex interfaces.
NDepend - This static analysis tool picks up where FxCop leaves off by providing visualizations and SQL-like code querying capabilities to identify the areas of the code that suffer from low cohesiveness, excessive complexity, and inappropriate dependencies. More importantly, it provides a big picture view of a code base that allows developers to identify and prioritize issues with the entire code base rather than just issues pertinent to the particular piece of functionality being implemented. This allows someone like an architect to effectively analyze a code-base without having to be intimately familiar with all of the details and provide objective evidence to backup any recommendations for making changes to the code.

Although I firmly believe that static analysis tools provide the most cost effective gains in code quality and should form the first line of defense in any shop's quest for code quality, I do think that manual reviews should still figure into the equation. Tools will never be able to verify whether code is soluble or that code logically satisfies the user's intent or that IT shared assets are effectively being utilized. I even think that a Developer's Handbook can provide value as long as it remains a living document that offers light-weight guidance rather than trying to exhaustively cover low level details that are ultimately technology specific and subject to change.

For anyone interested in learning more about NDepend, I recommend starting with this old Hanselminutes podcast followed by this quick 3 minute online demo of the code querying language in NDepend. Expect more posts from me on this most excellent tool in the near future.

Let Me Introduce You to My Little Friend: Resharper's Unit-Test Runner

Russell Ball — Tue, 11 Sep 2007 00:31:19 GMT

I used Resharper's unit-test runner for the first time while spelunking WatiN last week and quickly became a fan. If unit-test runners were high school boys, here are a few reasons why the Resharper test runner would be dating the entire cheer leading squad while the NUnit test runner would have to bribe a distant, homely cousin in order to get a prom date.

Integrated IDE experience: I think NUnit is good for deployment scenarios when you just want to load the test dll and verify the environment, but using a unit-test runner that is not integrated into the IDE is too frustrating for me when it comes to normal development. I used NUnit exclusively for several years, but I've been too spoiled by TestDriven.NET, SharpDevelop, and even MSTest to want to waste any more cycles switching between applications.
Navigation between Tests and Code: Resharper lets you run tests by right-clicking anywhere in the test code (similar to TestDriven.NET) or by clicking on the 'gutter icons' to the left of the code window. You can navigate in the other direction by simply double-clicking on the test icon, a feature I often use when investigating why tests fail.
Ad-Hoc Test Groupings - The only way to organize groups of tests that can be run together in NUnit is through namespaces. This works fine for groupings that you could plan out ahead of time such as fast-running vs. slow-running tests or tests that share common dependencies. However, if you do have to run tests from different namespaces in the NUnit GUI, the only way to do it is to highlight and run one test at a time (you can't even use the ctl-click option to highlight multiple tests). MSTest made a step in the right direction with their test list by allowing you to simply drag tests from different namespaces into a list, however Microsoft made the mistake of only including this feature in the Team Suite edition (which costs $10,000 plus an primary appendage) and also only allowing a test to belong to one list at a time. Resharper solves this issue by allowing you to add any test to a Test Session and also by implementing the Ctl-Clicking functionality to highlight multiple tests. There are also filter buttons on the toolbar that allow you to do things like only rerun the tests that failed.
Test Validity Status Monitoring: If you have changed and rebuilt your code without rerunning your tests, little question marks appear over the red/yellow/green icons. Depending on the size of the project and your use of shortcut keys, rebuilding can become an automatic reflex like saving a document so it's nice to get a smart reminder about whether or not your last test results are still valid.

Let me conclude with a disclaimer that I am primarily talking about the best way to run test written with the NUnit framework. When choosing between frameworks, I am now an advocate of MBUnit for reasons I'll outline in a future post.

Now THIS is a War Room

Russell Ball — Wed, 05 Sep 2007 15:53:23 GMT

In the Agile community, a war room refers to a team room where developers work, customer meetings take place, and all projected related information is displayed. It is supposed to maximize communication and transparency into the health of the project.

I think one sign of a healthy agile project is evidence of some good fun. These are pictures taken from a war room at the bank where I am currently wrapping up my near six year tenure. These guys have obviously taken the war room to a whole new level.

Update: It turns out that this was a clever prank by Rob. He explains the context in his blog post. Knowing the team, which does like to have fun, I am guessing that they will keep their new little green team members as a permanent fixture of their war room.

WatiN and the IEDevToolbar: The Dynamic Duo of Automated Web Testing

Russell Ball — Sat, 01 Sep 2007 02:16:28 GMT

Several weeks ago I used WatiN (pronounced as What-in) to create a suite of automated GUI tests for an old internal ASP application that we were upgrading. Inspired by the popular ruby testing tool Watir, WatiN is an open source .NET wrapper around the Internet Explorer that you can reference in your test project and use to get access to all of the HTML elements on a web page. I was surpised by how easy it was to simulate a user typing text and clicking buttons and then check to see if the result page contained certain text. The following popular example code demonstrates how you would test a google search for WatiN.

[Test]
public void SearchForWatiNOnGoogle()
{
using (IE ie = new IE("http://www.google.com"))
{
  ie.TextField(Find.ByName("q")).TypeText("WatiN");
  ie.Button(Find.ByName("btnG")).Click();

  Assert.IsTrue(ie.ContainsText("WatiN"));
}
}

Today, I experimented with the IEDevToolbar for the first time and suddenly WatiN became much more compelling. By using the "Select Element By Click" feature under the Find menu in the IEDevToolbar, you can simply click on an element on a web page and instantly see all of its properties. Since most popular commercial sites have very large, chaotic source files with elements that don't always have ID or Name properties, this feature is indispensible when trying to figure out how to programmatically interact with the page. Below shows how I was able to retrieve the Starbucks stock price from CNN and create a test to ensure that it was above an acceptable amount.

Since the stock retrieval button on cnn.com was an image that had no name or ID, I used the Find.ByCustom() method to retrieve and click on the button using the className attribute.

This task would be even easier using the WatiN Test Recorder, which you can see in this flash demo. However, I still haven't been able to get the recorder to work properly on my Vista machine so I'm not a big fan of this tool yet. I will definitely keep an eye out for future versions though.