Terje Sandstrom

------ Visual Studio ALM MVP ----- (also see new blog at http://hermit.no)

  Home  |   Contact  |   Syndication    |   Login
  64 Posts | 1 Stories | 116 Comments | 0 Trackbacks

News

Subscribe

Delicious Save this on Delicious Visual Studio Feeds

¨

Tag Cloud


Archives

Image Galleries

Company stuff

Interesting bloggers

Interesting companies

Microsoft

Microsoft Norge

Microsoft Regional Directors

Microsoft Test

MSFT Blogs

MVP

NNUG

Other interesting stuff

TFS

Visual Studio

Download:  VS2012 Runsettingstemplate   VS2013 Runsettingstemplate

The VS2012/13 unit test feature can generate code coverage results.  It can do so for (nearly) any type of adapter you choose to use, MSTest, CPPTest (managed/native), XUnit and NUnit (but not Chutzpah (note 1)).

Assume you have a project with a set of unit tests included.  For this demonstration I have used multiple different test frameworks just to show that this applies to any of these frameworks.

You choose to analyze for code coverage:

image

 

The code under test is this:

image

All the tests we have tests the Add method, none tests the Subtract method, so the expected code coverage should be 50%.

The result we get is this:

image

The total code coverage is calculated to 88%.  We also see all the tests, which – not surprisingly – is at 100%. 

 

Some people find this to be just fine, and want the code coverage to include the tests.  I , for one, are not so keen on this.  I want to see the code coverage of the real production code, and would have expected a result of 50% in this case. 

One can argue in different ways here, but let us assume that you do agree with me – and you want to get this result more correct :-)

We then need to exclude the tests from the code coverage results, and there are two ways to do this.

1) Add an ExcludeFromCodeCoverage attribute to the test classes

2) Add and enable a runsettings file to the solution. 

 

Option 1) will be tedious to use when the number of tests goes up, so we will go for Option 2).

After googling “runsettings” you will find this article, which tells you to copy settings information from that post into an emtpy runsettings file.  That is hard work!  And, if you use those settings, and you use  a 3rd party test adapter you will find more stuff in your test coverage results than you really bargained for.  (In one case I found the testadapters there!) We need to sweeten up this file.

And, we don’t want to do this manually each time.

Solution:  Install the Runsettings solution Item template from here:  VS2012 Runsettingstemplate   VS2013 Runsettingstemplate

Now, right click your Solution, choose Add Item:

image

Select the runsettings:

Your solution will get this added:

SNAGHTML216dc13b

Since we have added the default stuff and also the excludes for the unit testing in this, all you now need to do is to enable the use of this from the Test Menu:

image

You select the file using the “Select Test Settings File” and then you enable it by ensuring it has been checked.

 

Now when we run the Test Coverage we get this very nice result:

image

Compared to the results above, it is less clutter, as the irrelevant test code is not included in the results.  The resulting numbers are also more correct, imho.

 

Removing test code based on file exclusions

Most often your test code is placed in projects and assemblies suffixed with “Test”.  You can then use this as an exclusion criteria.  In that case you must open the runsettings file and add that criteria to the list.

Note, you don’t need to do this if your test files only contains attributed test classes, then the default runsettings file from the extension above will handle them.

Find  the section in the file named ModulePaths and add the relevant pattern there under the Exclude section.  Note that the patterns are Regular Expressions, and that special characters like ‘.’ and ‘*’may have special meanings. Test the exclusion properly.  See “Regular expressions” in this MSDN post.

image

 

 

Handling special test code

If you have code in your test project that is not covered by attributes, that code can be excluded by adding the ExcludeFromCodeCoverageAttribute to that part of the code, either class or method.  This can f.e. be stub classes or mocking classes, or common test code that does not have any of the other exclusion attributes.

 

TFS Build

Note that The TFS Build templates also have a property for setting the runsettingsfile.

image

This is the default setup, which means there is no coverage data.  If you select CodeCoverageEnabled( or …ForAspNetApps) then the standard codecoverage will be performed.  If you select a runsettings file, which must have been checked in to the repository, the Type goes to UserSpecified, and the specifications in the selected runsettings file takes effect.

image

Note that this don’t apply if you use the MSTest runner, but in TFS 2013, the VSTest runner is the default.

 

Note 1:   VS Test runner can’t find code coverage for javascript code, there is no dll’s to analyze, so it doesn’t work for Chutzpah, which handles QUnit andJasmine.

posted on Wednesday, December 4, 2013 8:49 PM

Feedback

# re: How to exclude code from Code Coverage in Visual Studio unit testing using runsettings 12/29/2014 8:37 PM art
One would think the MS developers would be smart enough to automate this whole process to begin with. But guess we are stuck with this archaic method...

# re: How to exclude code from Code Coverage in Visual Studio unit testing using runsettings 12/29/2014 8:42 PM Terje
Good point!

And yes, still not fixed, so I might be "forced" to add the same template for VS2015.


# re: How to exclude code from Code Coverage in Visual Studio unit testing using runsettings 5/7/2015 9:05 PM Josh
I'd rather add the template and have the flexibility. Great article! Very helpful.

# re: How to exclude code from Code Coverage in Visual Studio unit testing using runsettings 3/8/2016 6:57 AM Ravi
Hi, I have a simple console application- And covered all the unit test cases .If i run the code coverage only 70% is covered.Unable to cover code coverage for the variables/

Please find my code

static void Main(string[] args)
{
string firstName = string.Empty;
string lastName = string.Empty;

Console.WriteLine("Enter your First Name - ");
firstName = Console.ReadLine();

Console.WriteLine("Enter your Last Name - ");
lastName = Console.ReadLine();

Console.WriteLine(GetName(firstName.Trim(), lastName.Trim()));
Console.ReadKey();
}

public static string GetName(string firstName, string lastName)
{
if (!string.IsNullOrEmpty(firstName) && !string.IsNullOrEmpty(lastName))
return "Hi Mr. " + firstName + " " + lastName;
else
return "Please enter both firat and last Name";
}



# re: How to exclude code from Code Coverage in Visual Studio unit testing using runsettings 9/1/2016 11:58 PM Emma
I can't seem to get this extension to work using VS2015 with update 2. I installed the extension, restarted visual studio, and then double checked my extensions to make sure it was correctly installed, but when I right click on my solution and add new item, runsettings is not a listed item type.

# re: How to exclude code from Code Coverage in Visual Studio unit testing using runsettings 9/2/2016 12:06 AM Terje
Hi! I have noticed the same thing on occasions. But, first, you did install the 2015 version, right ? https://visualstudiogallery.msdn.microsoft.com/1cc3863b-f15f-4107-bb05-3586fd65540b There are seperate versions for each version of VS. Also, you say Update 2. I am currently using Update 3.1, and it seems to work correctly there. Another trick you can try, is to delete the MEF cache. See this post http://hermit.no/how-to-fix-visual-studio-loading-errors-using-ifix/

Post A Comment
Title:
Name:
Email:
Comment:
Verification: