Geeks With Blogs

@jakobehn

News

Microsoft Community Contributor Award 2011


Jakob Ehn Visual Studio ALM MVP

The build process template and custom activity described in this post is available here:
http://cid-ee034c9f620cd58d.office.live.com/self.aspx/BlogSamples/Inmeta%20TFS%20Build%20Sample.zip

Running code metrics has been available since VS 2008, but only from inside the IDE. Yesterday Microsoft finally released Visual Studio Code Metrics Power Tool 10.0, a command line tool that lets you run code metrics on your applications.  This means that it is now possible to perform code metrics analysis on the build server as part of your nightly/QA builds. In this post I will show how you can run the metrics command line tool from a build, and also a custom activity that reads the output and appends the results to the build log, and fails the build if the metric values exceeds certain (configurable) treshold values.

The code metrics tool analyzes all the methods in the assemblies, measuring cyclomatic complexity, class coupling, depth of inheritance and lines of code. Then it calculates a Maintainability Index from these values that is a measure of how maintanable this method is, between 0 (worst) and 100 (best). For information on how this value is calculated, see http://blogs.msdn.com/b/codeanalysis/archive/2007/11/20/maintainability-index-range-and-meaning.aspx. After this it aggregates the information and present it at the class, namespace and module level as well.


Running Metrics.exe in a build definition
Running the actual tool is easy, just use a InvokeProcess activity last in the Compile the Project sequence, reference the metrics.exe file and pass the correct arguments and you will end up with a result XML file in the drop directory. Here is how it is done in the attached build process template:

image

In the above sequence I first assign the path to the code metrics result file ([BinariesDirectory]\result.xml) to a variable called MetricsResultFile, which is then sent to the InvokeProcess activity in the Arguments property.
Here are the arguments for the InvokeProcess activity:

image

Note that we tell metrics.exe to analyze all assemblies located in the Binaries folder. You might want to do some more intelligent filtering here, you probably don’t want to analyze all 3rd party assemblies for example.
Note also the path to the metrics.exe, this is the default location when you install the Code Metrics power tool. You must of course install the power tool on all build servers.

Using the standard output logging (in the Handle Standard Output/Handle Error Output sections), we get the following output when running the build:

image

Integrating Code Metrics into the build
Having the results available next to the build result is nice, but we want to have results integrated in the build result itself, and also to affect the outcome of the build. The point of having QA builds that measure, for example, code metrics is to make it very clear how the code being built measures up to the standards of the project/company. Just having a XML file available in the drop location will not cause the developers to improve their code, but a (partially) failing build will! Ler

To do this, we need to write a custom activity that parses the metrics result file, logs it to the build log and fails the build if the values frfom the metrics is below/above some predefined treshold values.


The custom activity performs the following steps

  1. Parses the XML. I’m using Linq 2 XSD for this. Since the XML schema for the result file is available with the power tool, it is vey easy to generate code that lets you query the structure using standard Linq operators.
  2. Runs through the metric result hierarchy and logs the metrics for each level and also verifies maintainability index and the cyclomatic complexity against the treshold values. The treshold values are defined in the build process template and are sent in as arguments to the custom activity
  3. If the treshold limits are exceeded, the activity either fails or partially fails the current build.

For more information about the structure of the code metrics result file, read Cameron Skinner's post about it. It is very simple and easy to understand. I won’t go through the code of the custom activity here, since there is nothing special about it and it is available for download so you can look at it and play with it yourself.

The treshold values for Maintainability Index and Cyclomatic Complexity is defined in the build process template, and can be modified per build definition:

image

I have chosen the default values for these settings based on a post from my colleague Terje Sandström - Code Metrics - suggestions for approriate limits. When you think about it, this is quite an improvement compared to using code metrics inside the IDE, where the Red/Yellow/Green limits are fixed (and the default values are somewhat strange, see Terjes post for a discussion on this)


This is the first version of the code metrics integration with TFS 2010 Build, I will probably enhance the functionality and the logging (the “tree view” structure in the log becomes quite hard to read) soon.
I will also consider adding it to the Community TFS Build Extensions site when it becomes a bit more mature.

Another obvious improvement is to extend the data warehouse of TFS and push the metric results back to the warehouse and make it visible in the reports.

Posted on Sunday, January 30, 2011 8:39 PM TFS , Team Build , Visual Studio 2010 | Back to top


Comments on this post: Integrating Code Metrics in TFS 2010 Build

# re: Integrating Code Metrics in TFS 2010 Build
Requesting Gravatar...
Hi,

Great start, however the sample code appears to be missing some things, such as the definition of the Metric class as well as the CodeMetricsReport.Load methods. Are you missing files?

Left by Rick on Jan 31, 2011 5:29 AM

# re: Integrating Code Metrics in TFS 2010 Build
Requesting Gravatar...
@Rick: Sorry about that, I uploaded the full project now instead of the separate source files
Left by Jakob Ehn on Jan 31, 2011 9:32 AM

# re: Integrating Code Metrics in TFS 2010 Build
Requesting Gravatar...
Thanks a lot for this. However when I try to run my build process, I get could not find XML.Schema.Linq.dll eventhough I installed it in GAC. Would you please tell me what I'm missing? Any help would be much appreciated.
Left by kidambi on Feb 01, 2011 6:54 PM

# re: Integrating Code Metrics in TFS 2010 Build
Requesting Gravatar...
@Kidambi: You should not install it in the GAC, add it to source control, next to the custom activities assembly
Left by Jakob Ehn on Feb 01, 2011 9:22 PM

# re: Integrating Code Metrics in TFS 2010 Build
Requesting Gravatar...
Err, I've downloaded the sample, built it, but how do I deploy it to the build server so that I can set up a build to use it?
Left by Antony on Apr 20, 2011 11:24 AM

# re: Integrating Code Metrics in TFS 2010 Build
Requesting Gravatar...
Thanks for the sample, got it up and running, although I'm curious why you are processing metrics on all the levels, for example the cyclomatic value for the module, which is a aggregation of the namespaces. My particular assembly that is being evaluated has a module cyclomatic value of 111, which exceeds the threshold therefore the build fails. However, the members in the module don't exceed the threshold which my understanding is the only part that actually matters?
Left by Jason Cornwall on May 10, 2011 11:25 PM

# re: Integrating Code Metrics in TFS 2010 Build
Requesting Gravatar...
Thanks Jakob. The instructions are very clear and I've setup the metrics with help of your post and http://blogs.microsoft.co.il/blogs/shair/archive/2011/02/07/integrating-code-metrics-in-tfs-2010-build-wf-4-0.aspx

I got the outputs.xml file created but stuck in parsing the output into a readable format as part of Build log.

Could you please point me somewhere where I can get some more details on it.
Left by karthik on Sep 28, 2011 6:52 PM

# re: Integrating Code Metrics in TFS 2010 Build
Requesting Gravatar...
Do you have some additional advice how to push the data into warehouse?

best regards
Thorsten
Left by Thorsten on Oct 24, 2012 7:48 AM

# re: Integrating Code Metrics in TFS 2010 Build
Requesting Gravatar...
Hi,
This is great. Exactly what I was looking for. However I'm looking to download the source code and I could not get to it. Would you please point me from where I can download the code?

Thanks a lot.

Regards,

Kidambi
Left by Kidambi on Apr 04, 2014 4:43 PM

# re: Integrating Code Metrics in TFS 2010 Build
Requesting Gravatar...
@Kidimbi: This activity is now part of the Community TFS Build Extensions, and has been worked on since this blog post. Please check it out over at http://tfsbuildextensions.codeplex.com/

/Jakob
Left by Jakob Ehn on Apr 07, 2014 9:38 AM

# re: Integrating Code Metrics in TFS 2010 Build
Requesting Gravatar...
Hey,
Could you share the XSD you are using? I generated one from a sample report using the XSD tool, however when I try to deserialize using C#, the System.xml code throws an exception.
Left by Luis on Apr 08, 2014 11:00 PM

Your comment:
 (will show your gravatar)


Copyright © Jakob Ehn | Powered by: GeeksWithBlogs.net | Join free