Code Metrics - suggestions for appropriate limits

terje — Wed, 26 Nov 2008 03:20:49 GMT

I like Code Metrics. They give a certain "objective" evaluation of a piece of code. You can use it for yourself just to make it pinpoint potential trouble areas, or just some code where you were a bit sloppy - happens from time to time...... And when you're going to do a peer review, it's much better to point to some numbers instead of the "This code sucks....."-thing. So when it finally arrived inside Visual Studio I was delighted.

There are however a few things there which should be nice candidates for the next upgrade.

First, it doesn't work on Team Builds, and it won't be fixed in TFS 2008. The people I spoke to at the PDC said it would be integrated with team build in TFS 2010 !

Second, the levels they have for red/yellow/green limits are strange. It is only the Maintenance Index that "powers" these flags. At the method level they are fixed at <10 for Red and <20 for Yellow. The Maintenance index (MI) is a normalized variation of work done at Carnegie Mellon University, see http://blogs.msdn.com/fxcop/archive/2007/11/20/maintainability-index-range-and-meaning.aspx for information on it, and how they normalized the MI. The original metric had a maximum at 171. In VS2008 MI is defined to have a range from 0 to 100, so in principle it is scaled down by a factor of 100/171. In addition they did not allow negative values, which is just fine.

However, a study made by Hewlett Packard (using the original metrics), published in IEEE Computer back in 1994, where they analyzed several projects found that levels below 65 should be in the red, yellow from 65 to 85, and green beyond that. Normalizing these limits again, suggest values of 39 and 50 respectively, which is far higher than the 10/20 limits now in effect. My own experience have been to set these levels at 40 and 60, quite close to the HP findings. Code written with MI metrics values below 40 is rather nasty, so I believe these values still holds, even if the study is a few years back.

For the Cyclomatic complexity metric, which is a often used metric, the recommendations for red/yellow/green is Green <10, Yellow 10-15/20 and Red > 15/20. See this article. Some also says everything above 10 should be red.

The Class coupling is more complex, and it also depends on the Depth of Inheritance. I have not been able to find any solid recommendations there, but it seems that values beyond 15 to 20 should be avoided.

It would be nice if the flags shown in Visual Studio Code Metrics window could have customized limits - put them into a configuration file! First, they are fixed in code down in a method IconForMaintainabilityIndex in the CodeMetricsBranch class of the Microsoft.VisualStudio.Metrics.Package.UI. However, we can live without yellow and red flags, as long as the appropriate levels are known.

The table below summarizes this:

	Maintainability Index	Cyclomatic Complexity	Class Coupling
Green	> 60	< 10	< 20
Yellow	40 - 60	10 - 15
Red	< 40	> 15	> 20

Note that the values applies to the method level !

Hiding generated code from Code Analysis, Metrics and Test Coverage

terje — Mon, 10 Nov 2008 09:09:41 GMT

When I do either Code Analysis, Code Metrics or looking at Code Coverage results, I don't want to have any generated code affecting the results. It just confuses the numbers, and I do not really care how generated code looks - it should just be invisible.

Generated code appears several places, code is generated by any of the multitude of wizards and designers in Visual Studio, or it may be generated by a 3rd part tool or generated by a self-written tool.

There exist an attribute which, if attached to the class or method, is intended to hide the code from these analyses. However, it fails to do so for the Code Coverage. The attribute is the GeneratedCodeAttribute, and it is intended to be used for any tools generated code. See http://blogs.msdn.com/fxcop/archive/2007/04/27/correct-usage-of-the-compilergeneratedattribute-and-the-generatedcodeattribute.aspx for more information on the correct usage. It can be used both at class level and method level.

However, the Code Coverage doesn't abide by these rules, as also reported in a two bug reports http://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=338895 and http://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=349243. The latter indicate they will not fix this bug, so please go in a vote for a fix!

As a workaround there are two more attributes which can be used, the DebuggerNonUserCode and DebuggerHidden attributes. The former is a combination of the latter and the DebuggerStepThrough attribute. The DebuggerNonUserCode attribute can be used on both class and method level.

By using this attribute in addition on generated code, it will not show up in the Code Coverage either.

The table below summarizes this:

	GeneratedCodeAttribute	DebuggerNonUserCode DebuggerHidden
Code Metrics	Excludes code	No Effect
Code Analysis	Excludes code	No Effect
Code Coverage	No Effect	Excludes code

Further, Microsoft seems to have "forgotten" to add these attributes to some of their own generated code, among them is the Winform generated code. The MSDataSetGenerator adds the GeneratedCode attrbute, but not the DebuggerNonUserCode attribute. (If the Code Coverage had abided by the rules, this would not have been a problem)

For Winform generated code, it is safe to add these attributes afterwards. The generated file will not be rewritten, just changed, when you add more or edit the controls. Just add them as shown below:

        [DebuggerNonUserCode]
        [GeneratedCode("Winform Designer","VS2008 SP1")]
        private void InitializeComponent()

The MSDataSetGenerator is not so forgiving, since the file easily can be rewritten. However, all the classes are partial, so by adding another manual file, and using the attributes there, most of the DataSet can be removed from the Code Coverage. Adding one dataset with one table generates a lot of classes, the code below shows what is needed to eliminate a dataset and its tableadapter for a table named ErrorLog.

namespace WindowsFormsApplication10

    [DebuggerNonUserCode]

    public partial class DataSet1

        [DebuggerNonUserCode]

        public partial class ErrorLogDataTable

        [DebuggerNonUserCode]

        public partial class ErrorLogRow

    namespace DataSet1TableAdapters

        [DebuggerNonUserCode]

        public partial class ErrorLogTableAdapter

        [DebuggerNonUserCode]

        public partial class TableAdapterManager

In my little test case, code coverage changed from 2,54% to 10% just by removing the DataSet1 alone.

Under the Properties folder (for a WinForm app), the Resources class is decorated with both attributes (Hurrah!) (luckily since it's not partial....) whereas the Settings class only have the GeneratedCode attribute. (Why are these two different ?). The Settings.Designer.cs file is regenerated each time you change anything in the settings designer, but it is partial, so the same trick as above with the DataSet can be used - create a new file, with the same partial class declaration, and add the DebuggerNonUserCode to it.

In a later post I'll try to make an even more comprehensive list of which generators work and which doesn't, in this respect, and outline the workarounds for these.

Hopefully - all of these will be fixed in Visual Studio 2010.