Geeks With Blogs
// ThomasWeller C#/.NET software development, software integrity, life as a freelancer, and all the rest

This is the second of a three part series that deals with the issue of faking test data in the context of a legacy app that was built with Microsoft's Entity Framework (EF) on top of an MS SQL Server database – a scenario that can be found very often. Please read the first part for a description of the sample application, a discussion of some general aspects of unit testing in a database context, and of some more specific aspects of the here discussed EF/MSSQL combination.


Lately, I wondered how you would ‘mock’ the data layer of a legacy application, when this data layer is made up of an MS Entity Framework (EF) model in team with a MS SQL Server database. The question originally came up in the context of how you could enable higher-level integration tests (automated UI tests, to be exact) for a legacy application that uses this EF/MSSQL combo as its data store mechanism – a not so uncommon scenario.

The question sparked my interest, so I decided to dive into it a bit deeper. What I've found out is, in short, that it's not very easy and straightforward to do it – but it can be done. The two strategies that are best suited to fit the bill involve using either the (commercial) Typemock Isolator tool or the (free) NDbUnit framework. This post will present the Typemock approach...

When it comes to mocking and faking, Typemock is something like the swiss army knife in a tester's toolbelt. It's functionally comparable to MS Moles, in that there aren't any limits in what it can do (it's really Mocking the Unmockable). You can easily write all kinds of test doubles for components that you otherwise could not fake: Static classes, Framework classes, Sealed classes, Non-virtual members and so on... The big difference to Moles however is that Typemock has a well-crafted API and generally is so much easier to handle, understand, learn, and remember that this alone would justify the price difference.

Preparing the test data

As I have shown in the first part, the PersonRepository expects an instance of EF's ObjectContext class in its c'tor. This project-specific class is generated by the EF designer and contains, among (too) many other things, the conceptual entity model which is derived from the underlying database. (It's called SchoolEntities in our case.)

This marks the point where we can plug in the Typemock Isolator tool to isolate the system from the underlying database, faking the ObjectContext and thus having the opportunity to feed the PersonRepository class with self-provided data on a per-test basis as required.

The test fixture's Setup code will look like this, when using this approach:

[TestFixture, TestsOn(typeof(PersonRepository))]
[Description("Uses Typemock Isolator 2010 to fake the data for the applications EF data model. " +
             "Trial version can be downloaded from here: http://www.typemock.com/typemock-isolator-product3/")]
public class PersonRepositoryFixture
{
    #region Fields

    private SchoolEntities _schoolContext;
    private PersonRepository _personRepository;

    #endregion // Fields

    #region Setup/TearDown

    [SetUp]
    public void SetUp()
    {
        _schoolContext = Isolate.Fake.Instance<SchoolEntities>();
        _personRepository = new PersonRepository(_schoolContext);
    }

    ...

Note that we've altered only one single line of code here, compared to the 'original' version from the previous part: The _schoolContext field is initialized with a Fake instance of the SchoolEntities class instead of the 'real' object (see here for a short clarification of the sometimes confusing terminology around mocks, fakes, stubs and the like).

What does it mean? Well, it means that our _schoolContext field actually references a true instance of the SchoolEntities class, and not some sort of proxy object. You can call all methods and properties on it, they will do nothing by default and will return the respective default value for value types, or another Fake instance for reference types - unless we explicitely tell Typemock to return some custom, pre-defined values for us (or call some custom code on our behalf). And that's exactly what we need here to isolate the system under test from its EF data model (and its database, consequently).

To accomplish this, we first define a helper class to hold the test data that we will use (and also mimick other required database properties like e.g. referential constraints). It will act as a kind of 'in-memory database':

static class FakeDataHolder
{
    #region Fields

    // The xxxData collections serve as an internal cache. They represent the database,
    // are filled during construction  and not altered thereafter.
    private static readonly ReadOnlyCollection<Person> PersonData;
    private static readonly ReadOnlyCollection<OfficeAssignment> OfficeAssignmentData;

    // The xxxList collections mirror the xxxData collections. They represent a working copy of the data,
    // which is exposed through properties to the outside and can be reset via the 'Reset()' method.
    private static readonly List<Person> PersonList = new List<Person>();
    private static readonly List<OfficeAssignment> OfficeAssignmentList = new List<OfficeAssignment>();

    #endregion // Fields

    #region Properties

    public static IQueryable<Person> Persons
    {
        get { return PersonList.AsQueryable(); }
    }

    public static IQueryable<OfficeAssignment> OfficeAssignments
    {
        get { return OfficeAssignmentList.AsQueryable(); }
    }

    #endregion // Properties

    #region Construction

    static FakeDataHolder()
    { 
        PersonData = new List<Person>
            {
                new Person {PersonID = 1, FirstName = "Kim", LastName = "Abercrombie"},
                new Person {PersonID = 2, FirstName = "Gytis", LastName = "Barzdukas"},
                new Person {PersonID = 3, FirstName = "Peggy", LastName = "Justice"},
                new Person {PersonID = 4, FirstName = "Fadi", LastName = "Fakhouri"},
                new Person {PersonID = 5, FirstName = "Roger", LastName = "Harui"},
                new Person {PersonID = 6, FirstName = "Yan", LastName = "Li"},
                new Person {PersonID = 7, FirstName = "Laura", LastName = "Norman"},
                new Person {PersonID = 8, FirstName = "Nino", LastName = "Olivotto"},
                new Person {PersonID = 9, FirstName = "Wayne", LastName = "Tang"},
                new Person {PersonID = 10, FirstName = "Meredith", LastName = "Alonso"},
                new Person {PersonID = 11, FirstName = "Sophia", LastName = "Lopez"},
                new Person {PersonID = 12, FirstName = "Meredith", LastName = "Browning"},
                new Person {PersonID = 13, FirstName = "Arturo", LastName = "Anand"},
                new Person {PersonID = 14, FirstName = "Alexandra", LastName = "Walker"},
                new Person {PersonID = 15, FirstName = "Carson", LastName = "Powell"},
                new Person {PersonID = 16, FirstName = "Damien", LastName = "Jai"},
                new Person {PersonID = 17, FirstName = "Robyn", LastName = "Carlson"},
                new Person {PersonID = 18, FirstName = "Roger", LastName = "Zheng"},
                new Person {PersonID = 19, FirstName = "Carson", LastName = "Bryant"},
                new Person {PersonID = 20, FirstName = "Robyn", LastName = "Suarez"},
                new Person {PersonID = 21, FirstName = "Roger", LastName = "Holt"},
                new Person {PersonID = 22, FirstName = "Carson", LastName = "Alexander"},
                new Person {PersonID = 23, FirstName = "Isaiah", LastName = "Morgan"},
                new Person {PersonID = 24, FirstName = "Randall", LastName = "Martin"},
                new Person {PersonID = 25, FirstName = "Candace", LastName = "Kapoor"},
                new Person {PersonID = 26, FirstName = "Cody", LastName = "Rogers"},
                new Person {PersonID = 27, FirstName = "Stacy", LastName = "Serrano"},
                new Person {PersonID = 28, FirstName = "Anthony", LastName = "White"},
                new Person {PersonID = 29, FirstName = "Rachel", LastName = "Griffin"},
                new Person {PersonID = 30, FirstName = "Alicia", LastName = "Shan"},
                new Person {PersonID = 31, FirstName = "Jasmine", LastName = "Stewart"},
                new Person {PersonID = 32, FirstName = "Kristen", LastName = "Xu"},
                new Person {PersonID = 33, FirstName = "Erica", LastName = "Gao"},
                new Person {PersonID = 34, FirstName = "Roger", LastName = "Van Houten"}
            }
            .AsReadOnly(); 

        OfficeAssignmentData = new List<OfficeAssignment>
            {
                new OfficeAssignment { InstructorID = 18, Location = "143 Smith" }
            }
            .AsReadOnly();
    }

    #endregion // Construction

    #region Operations

    public static void Reset()
    {
        PersonList.Clear();
        PersonList.AddRange(PersonData);

        OfficeAssignmentList.Clear();
        OfficeAssignmentList.AddRange(OfficeAssignmentData);
    }

    public static void AddPerson(Person person)
    {
        PersonList.Add(person);
    }

    public static void DeletePerson(Person person)
    {
        if (OfficeAssignmentList.Any(@a => @a.InstructorID == person.PersonID))
        {
            throw new InvalidOperationException("FK_OfficeAssignment_Person");
        }

        PersonList.Remove(person);
    }

    #endregion // Operations

} // class FakedataHolder

The second step then would be to make our test fixture's _schoolContext field return these values (or call the appropriate FakeDataHolder methods, respectively). For this purpose, we can declare a helper method like this in the PersonRepositoryFixture class:

private void FakeData()
{
    FakeDataHolder.Reset(); 

    Isolate.WhenCalled(() => _schoolContext.People) 
           .WillReturnCollectionValuesOf(FakeDataHolder.Persons);
    Isolate.WhenCalled(() => _schoolContext.People.AddObject(null))
           .DoInstead(@ctx => FakeDataHolder.AddPerson(@ctx.Parameters[0] as Person));
    Isolate.WhenCalled(() => _schoolContext.People.DeleteObject(null))
           .DoInstead(@ctx => FakeDataHolder.DeletePerson(@ctx.Parameters[0] as Person));
}

I won't go into the details of Typemock's API syntax here (you may consult the Online Documentation for this, if you are interested), but generally I like it because it's very easy to read and understand without oversimplifying things. Or, in other words: I don't have to explain much here, the code itself tells it all...

Executing some basic tests

Here are some of the possible tests that can be written with the above preparations in place:

[Test, Isolated, MultipleAsserts, TestsOn("PersonRepository.GetNameList")]
public void GetNameList_ListOrdering_ReturnsTheExpectedFullNames()
{
    FakeData();

    List<string> names =
        _personRepository.GetNameList(NameOrdering.List);

    Assert.Count(34, names);
    Assert.AreEqual("Abercrombie, Kim", names.First());
    Assert.AreEqual("Zheng, Roger", names.Last());
}

[Test, MultipleAsserts, TestsOn("PersonRepository.GetNameList")]
public void GetNameList_NormalOrdering_ReturnsTheExpectedFullNames()
{
    FakeData();

    List<string> names =
        _personRepository.GetNameList(NameOrdering.Normal);

    Assert.Count(34, names);
    Assert.AreEqual("Alexandra Walker", names.First());
    Assert.AreEqual("Yan Li", names.Last());
}

[Test, Isolated, TestsOn("PersonRepository.AddPerson")]
public void AddPerson_CalledOnce_IncreasesCountByOne()
{
    FakeData();

    int count = _personRepository.Count;

    _personRepository.AddPerson(new Person { FirstName = "Thomas", LastName = "Weller" });

    Assert.AreEqual(count + 1, _personRepository.Count);
}

[Test, Isolated, TestsOn("PersonRepository.RemovePerson")]
public void RemovePerson_CalledOnce_DecreasesCountByOne()
{
    FakeData();

    int count = _personRepository.Count;

    _personRepository.RemovePerson(new Person { PersonID = 33 });

    Assert.AreEqual(count - 1, _personRepository.Count);
}

[Test, Isolated, TestsOn("PersonRepository.RemovePerson")]
public void RemovePerson_ForWhomAnOfficeAssignmentExists_Throws()
{
    FakeData();

    Assert.Throws<RepositoryException>(() =>
        _personRepository.RemovePerson(new Person { PersonID = 18 }));
}

I kept these tests deliberately simple to underpin the fact that Typemock-based tests don't need to be much different from non-faked tests, and also to make them comparable to the other options that are discussed here (the NDbUnit approach and the non-faked version, running your tests directly against a real instance of MS SQL Server, namely).

But wait, what if things are becoming somewhat more complex?

Admittedly (and intentionally), the above tests cover only some very simple test cases. This may be enough for some of your test scenarios: the 'normal' use cases, which may make up something well above 95% of what your application effectively is doing in production ("it just works"), but is easily covered with a few simple tests. The more interesting and important bits are the exceptional situations; the corner cases, where something unexpected happens. This is what consumes most of the time and effort in software development: To make an application robust and stable, and make it react in a predictable, user-friendly way even under exceptional circumstances.

In such a situation (i.e. if you want to simulate some sort of exceptional condition), chances are that you will have to dive deep into the production code - or even some 3rd party code, if it is available - to find out what you will have to return from your fake. A (still very simple) test of this kind could be this, for example:

[Test, Isolated, MultipleAsserts, TestsOn("PersonRepository.GetCourseMembers")]
[Row(null, typeof(ArgumentNullException))]
[Row("", typeof(ArgumentException))]
[Row("NotExistingCourse", typeof(ArgumentException))]
public void GetCourseMembers_WhenGivenVariousInvalidValues_Throws(string courseTitle, Type expectedInnerExceptionType)
{
    // Generally return false from the expression '_schoolContext.Courses.Any(...)'
    Isolate.WhenCalled(() => _schoolContext.Courses.Any(null))
           .WillReturn(false);

    var exception = Assert.Throws<RepositoryException>(() =>
                                _personRepository.GetCourseMembers(courseTitle));
    Assert.IsInstanceOfType(expectedInnerExceptionType, exception.InnerException);
}

This looks quite trivial, but it took me some minutes to understand that it's the Any() extension method that I had to fake here, and I also had to slightly refactor the FakeDataHolder() helper class to make this work. All in all, not too big a deal, you may say. True, but keep in mind that small things like that can easily add up to a huge amount of additional work (that your manager might not be willing to pay for)...

The other situation in which using Typemock becomes somewhat more laborious is when you have to perform a lot of preparational work of the above described kind to simulate a regular scenario, which you would otherwise (when running against a real database) just take for granted, without thinking much about it. Look at this test, for example:

[Test, Isolated, MultipleAsserts, TestsOn("PersonRepository.GetCourseMembers")]
public void GetCourseMembers_WhenGivenAnExistingCourse_ReturnsListOfStudents()
{
    // Avoid the ArgumentNullException
    Isolate.WhenCalled(() => _schoolContext.Courses.Any(null))
           .WillReturn(true);
    // Always return true from the expression 'Person.StudentGrades.Any(...)'
    var fakePerson = Isolate.Fake.Instance<Person>();
    Isolate.WhenCalled(() => fakePerson.StudentGrades.Any(null))
           .WillReturn(true);
    Isolate.Swap.AllInstances<Person>()
           .With(fakePerson);
    // Return the expected list of persons
    Isolate.WhenCalled(() => _schoolContext.People)
           .WillReturnCollectionValuesOf(new List<Person>
                                            {
                                                new Person {PersonID = 9, FirstName = "Wayne", LastName = "Tang"},
                                                new Person {PersonID = 10, FirstName = "Meredith", LastName = "Alonso"},
                                                new Person {PersonID = 11, FirstName = "Sophia", LastName = "Lopez"},
                                                new Person {PersonID = 12, FirstName = "Meredith", LastName = "Browning"},
                                                new Person {PersonID = 14, FirstName = "Alexandra", LastName = "Walker"},
                                                new Person {PersonID = 22, FirstName = "Carson", LastName = "Alexander"}
                                            }
                                            .AsQueryable());

    List<Person> persons = _personRepository.GetCourseMembers("Macroeconomics");

    Assert.Count(6, persons);
    Assert.ForAll(
        persons,
        @p => new[] { 9, 10, 11, 12, 14, 22 }.Contains(@p.PersonID),
        "Person has none of the expected IDs.");
}

The test asserts that, when asking for the members of a specific course, we actually are provided with the expected list, if we have set up the underlying ObjectContext fake to return the correct data. (One word about the Isolate.Swap.AllInstances() expression here: It instructs Typemock to replace all future instances of the Person class with the here defined fakePerson object, whenever the code under test should create a new Person instance.) See how the 'Arrange' part of the test blows up and also requires quite some intimate knowledge about the code under test, in order to provide the required return values (I've heard this once being called Wishful mocking...)?

Sometimes, especially when dealing with a legacy codebase that you cannot change, this is the only option at all to put such code under test, but it easily can mean that you will have to do a lot of work, if you will have to account for all kinds of database relations, triggers, check constraints, stored procedures and stuff like that. How much, will largely depend on the complexity of the underlying database (and of course also on the complexity of the code under test)...

Conclusion

Typemock is a powerful and flexible tool to handle the here described EF/MSSQL scenario. It's blazingly fast, fully isolates your system under test from any external data source, and lets you quickly and easily adapt/refactor your test code if you need to. All your test data are defined directly in the test code, so chances are high that you get immediate feedback in form of a compiler error when you have changed your database schema (and consequently the EF conceptual model) for some reason.

The downsides are more practical in nature: As I illustrated above, it can be a lot of work to fake certain scenarios with Typemock, so be prepared that it may not always be straightforward and easy to author tests with the help of the tool (but there will be a rich reward for your efforts...). Also, Typemock is a commercial tool,which means that it does not come for free in financial terms. You will have to make the explicit decision at some point whether it's worth its price in your specific situation or not.

Whereas this part of the article series has shown how the Typemock Isolator tool can be used to deal with a scenario where an application sits on top of an MS Entity Framework class model, the next part will accomplish the same with the NDbUnit framework.


The sample solution

A sample solution (VS 2010) with the code from this article series is available via my Bitbucket account from here (Bitbucket is a hosting site for Mercurial repositories. The repositories may also be accessed with the Git and Subversion SCMs - consult the documentation for details. In addition, it is possible to download the solution simply as a zipped archive – via the 'get source' button on the very right.). The solution contains some more tests against the PersonRepository class, which are not shown here. Also, it contains database scripts to create and fill the School sample database.

To compile and run, the solution expects the Gallio/MbUnit framework to be installed (which is free and can be downloaded from here), the NDbUnit framework (which is also free and can be downloaded from here), and the Typemock Isolator tool (a fully functional 30day-trial is available here). Moreover, you will need an instance of the Microsoft SQL Server DBMS, and you will have to adapt the connection strings in the test projects App.config files accordingly.

 

kickit
shoutit
delicious facebook digg reddit linkedin stumbleupon technorati mrwong yahoo google-48x48 twitter email favorites
Posted on Tuesday, November 8, 2011 7:27 AM Unit Testing/TDD , Automation | Back to top


Comments on this post: Testing Entity Framework applications, pt. 2: Typemock

# re: Testing Entity Framework applications, pt. 2: Typemock
Requesting Gravatar...
What exactly are you trying to test? Unit testing is for testing business logic or code that will frequently change. How often are you going to modify your repository implementation? It looks like you want to do an integration test. In that case there is no reason not to use a real database instance. The goal of an integration test is to make sure components work together. By faking the DB you're missing a critical component.

Besides you could just fire up your app and see if you can log in/view data. If you can, it works. If it doesn't obviously it's broken. This would be visible to anyone doing development on the app and is not necessary to automate.

Just my 2 cents.
Left by Ryan on Nov 09, 2011 7:44 AM

# re: Testing Entity Framework applications, pt. 2: Typemock
Requesting Gravatar...
I'll just go ahead and add that unit testing things like sorting (unless it's business critical domain logic) is pretty useless. Why not just keep the code as simple as possible?

var sortedStudents = DB
.GetPeople()
.Where(x => x.Type == Student)
.OrderBy(x => x.Name);

Then you (or a pair programmer or QA guy) can look at the code and see very clearly what it does WAY FASTER than you could ever write a quality test. Just reading your sort order test cases took longer than understanding the code they test. You would have been better off just testing Person.GetFullName() because that was the only difference between the two cases. (But again, it's trivial code anyway...)

With fragile tests in place, when you want to change the sort style (for whatever unknown future reason), you'd have to update tests too. If you have any lazy programmers on your team (we all do), they won't do it, they'll nuke the test, or use the test as an excuse NOT to adapt to some future requirement change. In the long term it just makes the project more expensive.
Left by Ryan on Nov 09, 2011 8:00 AM

# re: Testing Entity Framework applications, pt. 2: Typemock
Requesting Gravatar...
Ryan,

1.
I'm not trying to test anything. If you had read the blog(s), you'd know that. I'm disscussing some technologies to inject some test data into a system to open it for other tests.

2.
Why should this be fragile? That's sounds to me like unreflected stuff that you just have read in some textbook, and now you think you've found the silver bullet.

3.
'Just do X and see if it works' is exactly the thing that more experienced developers want to avoid.

You really should read the blog post(s) instead of telling things that are long found to be wrong...

- Thomas
Left by Thomas Weller on Nov 09, 2011 10:37 AM

# re: Testing Entity Framework applications, pt. 2: Typemock
Requesting Gravatar...
I do agree with Ryan. It's hard to tell what you are trying to do. And some of your tests are pointless, for instance, take this test:

[Test, Isolated, TestsOn("PersonRepository.RemovePerson")]
public void RemovePerson_CalledOnce_DecreasesCountByOne()
{
FakeData();
int count = _personRepository.Count;
_personRepository.RemovePerson(new Person { PersonID = 33 });

Assert.AreEqual(count - 1, _personRepository.Count);
}

the assert on count doesn't say anything. for all we know, any person could have been deleted.

I shouldn't have to read all your posts to figure out what you are trying to do. Or if that are required: add that as a disclaimer at top of your post (and linking to the mandatory reads).

You should be happy that readers take the time commenting. It helps you improve your posts.
Left by jgauffin on Nov 09, 2011 3:33 PM

# re: Testing Entity Framework applications, pt. 2: Typemock
Requesting Gravatar...
jgauffin,

I don't know how else I could say this: This post is not about testing! So your comment about the presented tests is pointless. They just serve as usage examples.

The post states this as clear as possible, and I can't help if you're not reading it...

- Thomas
Left by Thomas Weller on Nov 09, 2011 4:04 PM

# re: Testing Entity Framework applications, pt. 2: Typemock
Requesting Gravatar...
It's rather funny that you think that the post is clear when the only two taking time commenting it says that it's not.
Left by jgauffin on Nov 09, 2011 4:25 PM

# re: Testing Entity Framework applications, pt. 2: Typemock
Requesting Gravatar...


Thank you, but no need to worry about the understandability of this post.

Sorry, but the problem is definitely not on my side...
Left by Thomas Weller on Nov 09, 2011 5:23 PM

# re: Testing Entity Framework applications, pt. 2: Typemock
Requesting Gravatar...
"This post is not about testing!"

I think the title says otherwise?

Sorry, couldn't resist :P
Left by Wes on Nov 09, 2011 5:53 PM

# re: Testing Entity Framework applications, pt. 2: Typemock
Requesting Gravatar...
Wes,

the title is 'Testing xxx apps', but not 'Testing xxx'...
Left by Thomas Weller on Nov 09, 2011 6:08 PM

Your comment:
 (will show your gravatar)
 


Copyright © Thomas Weller | Powered by: GeeksWithBlogs.net | Join free