Dylan Smith

ALM / Architecture / TFS

  Home  |   Contact  |   Syndication    |   Login
  71 Posts | 0 Stories | 114 Comments | 29 Trackbacks

News



Archives

Post Categories

Blogs I Read

There's been an interesting discussion going on in the TDD mailing list, discussing the benefits (or lack of) of using mock objects and mocking frameworks. The discussion focuses on doing top-down development, where you tackle the high-level code first and during the implementation you “discover“ what lower-level supporting code is required.  Instead of diving down and writing this low-level code immediately, you defer it's implementation and continue writing the high-level code having it call out to the lower-level classes/methods even though they don't exist yet.  This helps drive the design of the lower-level code by creating “specs“ for how you wish it to be called and behave.

The obvious problem is if you are using classes/methods in your code that don't exist yet, then you have no chance of having your code compile and/or work until the lower level code is also implemented.  Well with TDD we want to have passing tests, and have them often.  Since you're writing the high-level implementation code already, this implies you already have a test in place to exercise it.  In a complex system it's not feasable to wait until the low-level code is implemented before achieving the passing test for the high-level code.  The idea is we want to write the implementation in small increments.  We need some way to defer the implementation of this low-level code, so that we can test the high-level code on it's own.

There are a number of solutions to this, the two that are at issue here are what I will refer to as mocks and stubs.  I say that's what I will refer to them as because in my limited experience it seems the terms (especially mock) are used rather loosely.  When I say mock I am referring to using one of the mocking frameworks (Rhino Mocks, NMock, EasyMock.Net, etc) to generate a special mock object to take the place of the real object.  The mock is created as part of the test, and the return value is hardcoded into the test.  This allows your high-level code to make calls out to a low-level class/method, and the high level code is still testable in isolation by replacing the low-level code with a mock for the purposes of testing. 

When I say stub I am referring to creating a skeleton implementation of the actual class in question, and writing some dummy code that returns a hardcoded value.  For some more background you can read Martin Fowler's article: Mocks Aren't Stubs.

To highlight the differences in code, I'll use an example of an Employee class that has to implement the high-level method CalculatePay.  This accepts a start and end date to use for the calculation.  To perform this calculation the code will need to retrieve the employees total labor over the period, the employees rate of pay, and a per-hour premium that is applied.  For this example, lets assume that there is a requirement that the premium can be either positive or negative. Following the separation of concerns principle, we realize that we will require additional methods which will be responsible for retrieving the total labor, retrieving the rate of pay, and retrieving the premium.  Lets assume that all this data is ultimately retrieved from a database somewhere.

Following is the version using mocks.  This code has the first test written, and the implementation to make the test pass.

[TestFixture]

public class EmployeeTests {

    private MockRepository _Mocks;

    private IEmployee _EmployeeMock;

 

    public EmployeeTests() { }

 

    [SetUp]

    public void Setup() {

        _Mocks = new MockRepository();

        _EmployeeMock = _Mocks.CreateMock<IEmployee>();

    }

 

    [Test]

    public void CalculatePayTest() {

        Employee emp = new Employee(1234);

        DateTime startDate = DateTime.Parse("1-Jul-2006");

        DateTime endDate = DateTime.Parse("31-Jul-2006");

 

        Expect.Call(_EmployeeMock.GetLabor(startDate, endDate)).Return(40);

        Expect.Call(_EmployeeMock.GetPayRate()).Return(48);

        Expect.Call(_EmployeeMock.GetPremium()).Return(2);

 

        emp.Target = _EmployeeMock;

 

        _Mocks.ReplayAll();

        int result = emp.CalculatePay(startDate, endDate);

        _Mocks.VerifyAll();

 

        Assert.AreEqual(2000, result);

    }

}

 

public interface IEmployee {

    int CalculatePay(DateTime startDate, DateTime endDate);

    int GetLabor(DateTime startDate, DateTime endDate);

    int GetPayRate();

    int GetPremium();

}

 

public class Employee:IEmployee{

    private IEmployee _Target;

    private int _EmployeeId;

 

    public IEmployee Target {

        get{return _Target;}

        set{_Target = value;}

    }

 

    public Employee(int employeeId) { _EmployeeId = employeeId; }

 

    public int CalculatePay(DateTime startDate, DateTime endDate) {

        int labor = _Target.GetLabor(startDate, endDate);

        int rate = _Target.GetPayRate();

        int premium = _Target.GetPremium();

 

        return labor * (rate + premium);

    }

 

    public int GetLabor(DateTime startDate, DateTime endDate) {

        return 0;

    }

 

    public int GetPayRate() {

        return 0;

    }

 

    public int GetPremium() {

        return 0;

    }

}

 

The same code using stubs would look something like this:

[TestFixture]

public class EmployeeTests {

    public EmployeeTests() { }

 

    [SetUp]

    public void Setup() { }

 

    [Test]

    public void CalculatePayTest() {

        Employee emp = new Employee(1234);

        DateTime startDate = DateTime.Parse("1-Jul-2006");

        DateTime endDate = DateTime.Parse("31-Jul-2006");

 

        int result = emp.CalculatePay(startDate, endDate);

 

        Assert.AreEqual(2000, result);

    }

}

 

public class Employee{

    private int _EmployeeId;

 

    public Employee(int employeeId) { _EmployeeId = employeeId; }

 

    public int CalculatePay(DateTime startDate, DateTime endDate) {

        int labor = GetLabor(startDate, endDate);

        int rate = GetPayRate();

        int premium = GetPremium();

 

        return labor * (rate + premium);

    }

 

    public int GetLabor(DateTime startDate, DateTime endDate) {

        return 40;

    }

 

    public int GetPayRate() {

        return 48;

    }

 

    public int GetPremium() {

        return 2;

    }

}

The first thing I notice is that the code with mocks is quite a bit larger.  The mocking code needs to explicitly define an interface for the Employee class to support mocking.  It also has to use Inversion Of Control / Dependency Injection via setter injection to provide a layer of indirection to support mocking.  The stub code does not require this extra complexity, at least not at this point in development.  The mocking version also requires extra test setup, and more work in the test itself to specify the expectations for the mocks.

Another possible strike against the mocking version is it may know too much about the test subject, and it's expectations may be too detailed.  For instance, If the CalculatePay() function were to change it's code so it retrieves the labor after the rate, this will cause the mocking test to fail.  The CalculatePay() will still be correct, and produce the correct result, it just goes about it in a different way.  This is an example of a test “knowing“ too much about it's test subject.  In theory, a test should only be testing that the test subject is producing the correct behaviour, not testing the details of it's implementation and how it goes about achieving that behaviour.

An advantage the mocking version has is it's more readable (in my opinion).  The data that is used in the non-mocking test is split up between the test code, and the stub functions that return the hardcoded values (40, 48, 2).  In the mocking version the data used by the test is all contained within the test code in the form of expectations on the mock.

To demonstrate another important difference, lets say that we now wanted to implement another test.  Let's say that we wanted to write a test that involved a negative premium. In this case, we are almost certain that the code will work without modification regardless of whether the premium is positive or negative.  But for the sake of argument, lets assume that we are not so certain and we want to write a test to verify that it will work under these circumstances.

If we were using the mock approach, this would be straightforward.  We could just write another test, and have the expectations on the mock return different values that included a negative premium.  But what if we are using the stub approach?  If we change the hardcoded return values to test with a negative premium then the first test will no longer pass.  One approach is to start writing tests against the GetLabor, GetPayRate, and GetPremium functions to drive the implementation of those.  Then we can come back and write further tests against the CalculatePay once the lower-level functions are implemented and tested.  That option isn't very attractive to me.  Now I am being pressured into testing/implementing the low-level code when maybe I'm not ready yet.  Perhaps there are more tests, and logic I wish to focus on in the high-level CalculatePay() function before I move on to the lower-level code.  With the mocking approach I have the freedom to focus on any portion of the code I want, and move on when I am ready.  This is a result of the mock objects isolating the test subject from the lower-level code it depends on.

Another advantage to the mocking approach is it can allow you more flexibility in the development process if you are working with a team.  Perhaps I am responsible for writing one chunk of code but somebody else on the team is responsible for some other piece of code that mine will depend upon and interact with.  In this situation it may not be feasable for me to start writing a “stubby“ implementation of the dependency because my colleague is working on it at the moment.  If mocks are used in the tests, this will allow me to test my piece of code independent of the dependencies that may be outside my realm of responsibility.

To summarize, it appears to me that mocking involves more work to implement, but the payoff is added flexibility to the developer in which tests you write and which implementations you can and should focus on.  It also improves the tests' readability.  The downside is your tests become more brittle a result of them knowing too much about the inner workings of the implementation.  Which approach is best?  I don't think there is a clear answer to that.  Both approaches are valid, and have their own distinct benefits.  I think that there is a tradeoff between the two, and hopefully this helped give you a better understanding of the tradeoffs so you can make a more informed decision which approach is best under your circumstances then next time you need to write some tests.

posted on Saturday, August 12, 2006 7:52 PM

Feedback

# re: When should you use Mocks vs Stubs? 8/12/2006 9:29 PM Ayende Rahien
Just to point out, this:
_EmployeeMock = (IEmployee)_Mocks.CreateMock<IEmployee>();

Can be also written as this:

_EmployeeMock = _Mocks.CreateMock<IEmployee>();

You don't need the cast if you are using the generic method.

# re: When should you use Mocks vs Stubs? 8/12/2006 9:48 PM Dylan
Thanks Ayende. Post has been edited.

# re: When should you use Mocks vs Stubs? 8/13/2006 12:39 PM Thomas Eyde
I would argue that your stub version is not done: You have not refactored away the fake values.

Here's an attempt to fullfil it:

1. The labour must be retrieved from somewhere. The example suggests there is a timesheet somewhere.

2. The TimeSheet is our stubbing target.

The result (some dead code removed):

[TestFixture]
public class EmployeeTests
{
[Test]
public void CalculatePayTest()
{
Employee emp = new Employee(new TimeSheet(40, 48, 2));
DateTime startDate = DateTime.Parse("1-Jul-2006");
DateTime endDate = DateTime.Parse("31-Jul-2006");

int result = emp.CalculatePay(startDate, endDate);

Assert.AreEqual(2000, result);
}
}

public class Employee
{
private TimeSheet sheet;

public Employee(TimeSheet sheet)
{
this.sheet = sheet;
}

public int CalculatePay(DateTime startDate, DateTime endDate)
{
int labor = sheet.GetLabour(startDate, endDate);
int rate = sheet.GetPayRate();
int premium = sheet.GetPremium();

return labor*(rate + premium);
}
}

public class TimeSheet
{
private int labour;
private int payRate;
private int premium;

public TimeSheet(int labour, int payRate, int premium)
{
this.labour = labour;
this.payRate = payRate;
this.premium = premium;
}

public int GetLabour(DateTime startDate, DateTime endDate)
{
return labour;
}

public int GetPayRate()
{
return payRate;
}

public int GetPremium()
{
return premium;
}
}


# re: When should you use Mocks vs Stubs? 11/2/2006 10:31 AM Dylan
Thomas,
I agree with you that the stub version isn't done; neither version is "done". However, both versions are at a green bar, so using TDD principles I shouldn't be writing new implementation code without a test. Certainly I can move the stubbing down to a lower level should I choose, but that is exactly what I'd like to avoid. I want the freedom to choose what piece of the code to focus on next. Maybe I am not interested in defining what the storage mechanism will be at this point in time, maybe I am more interested in fleshing out some of the payroll calculation logic.

I feel that mocking provides a greater degree of flexibility. With mocking I am free to write pretty much any test I can imagine on the high level code without being forced to implement anything farther down the stack. With stubs I am somewhat more restricted in what type of tests I can write without having to focus on the next layer of code.

Post A Comment
Title:
Name:
Email:
Comment:
Verification: