Geeks With Blogs

News




View Tarun Arora's profile on LinkedIn

profile for Tarun Arora at Stack Overflow, Q&A for professional and enthusiast programmers

Tarun Arora - Visual Studio ALM MVP ALM, Agile, Automation, Performance Testing, Software QA, Cloud, ...

 

It’s a Friday evening and you have just checked in that last bit of code, you are waiting for the build to go all green so that you could call it a day. Just then the build summary page comes back red on the test results section. The build summary page tells you that 1 of the test namely HelloWorld.Tests.UnitTest1.Sum_TwoIntNumbers_IncorrectResult_TestMethod has failed…

image

To know why the test has failed, you have no option but to click on View Test Results link which in turn downloads the trx file from the server so that you can see the error message and possibly identify the root cause of failure from the description error message. If the test list contains more than a 1000 tests you will notice that it takes a lot of time in downloading the test results locally. That’s probably not what you would want to do on a Friday evening…

image

In this blog post I’ll show you how to use the TFS API to write a simple utility to pull down the test result details programmatically and a bonus power tool for those who read the complete post, let’s get started…

A peek at what we will get to…

image

1. Connecting to TFS using the API

If you are new to the TFS API, refer to this blog post on getting started with the Visual Studio SDK.

The below snippet will help present a connect to tfs pop up to the user

private TfsTeamProjectCollection _tfs;
private string _selectedTeamProject;

public void ConnectToTfs()
{
       TeamProjectPicker tfsPP = new TeamProjectPicker(TeamProjectPickerMode.SingleProject, false);
       tfsPP.ShowDialog();
       this._tfs = tfsPP.SelectedTeamProjectCollection;
       this._selectedTeamProject = tfsPP.SelectedProjects[0].Name;
}

 

2. Get All Build Definitions for the selected Team Project

By using the IBuildService you can get access to the QueryBuildDefinition method which takes team project and returns the build definitions associated to the team project.

private IBuildDefinition[] GetAllBuildDefinitionsFromTheTeamProject(_selectTeamProject)
{
      _bs = _tfs.GetService<IBuildServer>();
      return _bs.QueryBuildDefinitions(_selectedTeamProject);
}

 

3. Get All Builds Associated to the Build Definition

Get All Builds in the build definition for specific build Definition specifications programmatically, this helps you narrow down the search and get the selected few filtered results, like in the snippet below, we specify the maximum number of builds to return, what build quality builds to return, etc.

// Get All Builds for the selected build definition 
private void cbBuildDef_SelectionChanged(object sender, SelectionChangedEventArgs e)
{
       var bdef = (((ComboBox)sender).SelectedItem) as IBuildDefinition;
       var def = _bs.CreateBuildDetailSpec(_selectedTeamProject);
       def.MaxBuildsPerDefinition = 10;
       def.QueryOrder = BuildQueryOrder.FinishTimeDescending;
       def.DefinitionSpec.Name = bdef.Name;
       def.Status = BuildStatus.All;
       var builds = _bs.QueryBuilds(def).Builds;

        cbBuild.ItemsSource = _bs.QueryBuilds(def).Builds;
        cbBuild.DisplayMemberPath = "BuildNumber";
        cbBuild.SelectedValuePath = "Uri";
        cbBuild.SelectedIndex = 0;
}

 

4. Get all the Test Results associated to a Build

I’ll be using the ITestManagementService to pull all test runs associated to a build. As you can see below by using the QueryResultsByOutcome I can query for tests that passed or failed or tests that errored. You could also use the QueryResultsByOwner to get test results by individual tfs users.

private void GetTestResult(Uri buildUri)
{
       _tms = _tfs.GetService<ITestManagementService>();
       var testRuns = _tms.GetTeamProject(_selectedTeamProject).TestRuns.ByBuild(buildUri);

        foreach (var testRun in testRuns)
        {
             lstTestRunDetails.Items.Add(string.Format("{0}", testRun.Title));
             lstTestRunDetails.Items.Add(string.Format("TestRunId: {0} | TestPlanId: {1}", testRun.Id, testRun.TestPlanId));
             lstTestRunDetails.Items.Add(string.Format("TestSettingsId: {0} | TestEnvironmentId {1} ", testRun.TestSettingsId, testRun.TestEnvironmentId));

             var totalTests = testRun.Statistics.TotalTests;

              foreach (var et in testRun.QueryResultsByOutcome(TestOutcome.Error))
              {
                    lstTestRunDetails.Items.Add(string.Format("{0}: {1} - {2}", et.Outcome, et.TestCaseTitle, et.ErrorMessage));
              }

              foreach (var tp in testRun.QueryResultsByOutcome(TestOutcome.Passed))
              {
                    lstTestRunDetails.Items.Add(string.Format("{0}: {1} ", tp.Outcome, tp.TestCaseTitle));
               }

                foreach (var tf in testRun.QueryResultsByOutcome(TestOutcome.Failed))
                {
                    lstTestRunDetails.Items.Add(string.Format("{0}: {1} - {2}", tf.Outcome, tf.TestCaseTitle, tf.ErrorMessage));
                }

                foreach (var tw in testRun.QueryResultsByOutcome(TestOutcome.Warning))
                {
                    lstTestRunDetails.Items.Add(string.Format("{0}: {1} - {2}", tw.Outcome, tw.TestCaseTitle, tw.ErrorMessage));
                }

                foreach (var ta in testRun.QueryResultsByOutcome(TestOutcome.Aborted))
                {
                    lstTestRunDetails.Items.Add(string.Format("{0}: {1} - {2}", ta.Outcome, ta.TestCaseTitle, ta.ErrorMessage));
                }

                foreach (var tb in testRun.QueryResultsByOutcome(TestOutcome.Blocked))
                {
                    lstTestRunDetails.Items.Add(string.Format("{0}: {1} - {2}", tb.Outcome, tb.TestCaseTitle, tb.ErrorMessage));
                }

                foreach (var ti in testRun.QueryResultsByOutcome(TestOutcome.Inconclusive))
                {
                    lstTestRunDetails.Items.Add(string.Format("{0}: {1} - {2}", ti.Outcome, ti.TestCaseTitle, ti.ErrorMessage));
                }

                foreach (var to in testRun.QueryResultsByOutcome(TestOutcome.Timeout))
                {
                    lstTestRunDetails.Items.Add(string.Format("{0}: {1} - {2}", to.Outcome, to.TestCaseTitle, to.ErrorMessage));
                }

                // Get the test results by user by passing in the Test Foundation Identity
                // testRun.QueryResultsByOwner(TeamFoundationIdentity);
            }

            if(testRuns.Count() == 0)
                lstTestRunDetails.Items.Add("No Test Results have been associated with the selected build");
 }

 

5. Putting it all together

Download the sample application – wpf, .net 4, you need to have vs sdk installed on your machine to run it.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Windows;
using System.Windows.Controls;
using System.Windows.Data;
using System.Windows.Documents;
using System.Windows.Input;
using System.Windows.Media;
using System.Windows.Media.Imaging;
using System.Windows.Navigation;
using System.Windows.Shapes;
using Microsoft.TeamFoundation.Build.Client;
using Microsoft.TeamFoundation.Client;
using Microsoft.TeamFoundation.TestManagement.Client;
using Microsoft.TeamFoundation.VersionControl.Client;

namespace TfsApiGetTestResultsAndDetailsByStatus
{
    /// <summary>
    /// Interaction logic for MainWindow.xaml
    /// </summary>
    public partial class MainWindow : Window
    {
        public MainWindow()
        {
            InitializeComponent();
        }

        private TfsTeamProjectCollection _tfs;
        private string _selectedTeamProject;
        private IBuildServer _bs;
        private VersionControlServer _vcs;
        private ITestManagementService _tms;

        public void ConnectToTfs()
        {
            TeamProjectPicker tfsPP = new TeamProjectPicker(TeamProjectPickerMode.SingleProject, false);
            tfsPP.ShowDialog();
            this._tfs = tfsPP.SelectedTeamProjectCollection;
            this._selectedTeamProject = tfsPP.SelectedProjects[0].Name;
        }

        private void btnConnect_Click(object sender, RoutedEventArgs e)
        {
            ConnectToTfs();
            _vcs = _tfs.GetService<VersionControlServer>();
            cbBuildDef.ItemsSource = GetAllBuildDefinitionsFromTheTeamProject();
            cbBuildDef.DisplayMemberPath = "Name";
            cbBuildDef.SelectedValuePath = "Uri";
            cbBuildDef.SelectedIndex = 0;
        }

        private IBuildDefinition[] GetAllBuildDefinitionsFromTheTeamProject()
        {
            _bs = _tfs.GetService<IBuildServer>();
            return _bs.QueryBuildDefinitions(_selectedTeamProject);
        }

        // Get All Builds for the selected build definition 
        private void cbBuildDef_SelectionChanged(object sender, SelectionChangedEventArgs e)
        {
            var bdef = (((ComboBox)sender).SelectedItem) as IBuildDefinition;
            var def = _bs.CreateBuildDetailSpec(_selectedTeamProject);
            def.MaxBuildsPerDefinition = 10;
            def.QueryOrder = BuildQueryOrder.FinishTimeDescending;
            def.DefinitionSpec.Name = bdef.Name;
            def.Status = BuildStatus.All;
            var builds = _bs.QueryBuilds(def).Builds;

            cbBuild.ItemsSource = _bs.QueryBuilds(def).Builds;
            cbBuild.DisplayMemberPath = "BuildNumber";
            cbBuild.SelectedValuePath = "Uri";
            cbBuild.SelectedIndex = 0;
        }

        private void cbBuild_SelectionChanged(object sender, SelectionChangedEventArgs e)
        {
            var build = (((ComboBox)sender).SelectedItem) as IBuildDetail;
            if (build == null && ((ComboBox)sender).Items.Count != 0)
            {
                build = ((ComboBox)sender).Items[0] as IBuildDetail;
                var def = _bs.CreateBuildDetailSpec(_selectedTeamProject);
                GetTestResult(build.Uri);
            }
        }

        private void GetTestResult(Uri buildUri)
        {
            _tms = _tfs.GetService<ITestManagementService>();
            var testRuns = _tms.GetTeamProject(_selectedTeamProject).TestRuns.ByBuild(buildUri);

            foreach (var testRun in testRuns)
            {
                lstTestRunDetails.Items.Add(string.Format("{0}", testRun.Title));
                lstTestRunDetails.Items.Add(string.Format("TestRunId: {0} | TestPlanId: {1}", testRun.Id, testRun.TestPlanId));
                lstTestRunDetails.Items.Add(string.Format("TestSettingsId: {0} | TestEnvironmentId {1} ", testRun.TestSettingsId, testRun.TestEnvironmentId));

                var totalTests = testRun.Statistics.TotalTests;

                foreach (var et in testRun.QueryResultsByOutcome(TestOutcome.Error))
                {
                    lstTestRunDetails.Items.Add(string.Format("{0}: {1} - {2}", et.Outcome, et.TestCaseTitle, et.ErrorMessage));
                }

                foreach (var tp in testRun.QueryResultsByOutcome(TestOutcome.Passed))
                {
                    lstTestRunDetails.Items.Add(string.Format("{0}: {1} ", tp.Outcome, tp.TestCaseTitle));
                }

                foreach (var tf in testRun.QueryResultsByOutcome(TestOutcome.Failed))
                {
                    lstTestRunDetails.Items.Add(string.Format("{0}: {1} - {2}", tf.Outcome, tf.TestCaseTitle, tf.ErrorMessage));
                }

                foreach (var tw in testRun.QueryResultsByOutcome(TestOutcome.Warning))
                {
                    lstTestRunDetails.Items.Add(string.Format("{0}: {1} - {2}", tw.Outcome, tw.TestCaseTitle, tw.ErrorMessage));
                }

                foreach (var ta in testRun.QueryResultsByOutcome(TestOutcome.Aborted))
                {
                    lstTestRunDetails.Items.Add(string.Format("{0}: {1} - {2}", ta.Outcome, ta.TestCaseTitle, ta.ErrorMessage));
                }

                foreach (var tb in testRun.QueryResultsByOutcome(TestOutcome.Blocked))
                {
                    lstTestRunDetails.Items.Add(string.Format("{0}: {1} - {2}", tb.Outcome, tb.TestCaseTitle, tb.ErrorMessage));
                }

                foreach (var ti in testRun.QueryResultsByOutcome(TestOutcome.Inconclusive))
                {
                    lstTestRunDetails.Items.Add(string.Format("{0}: {1} - {2}", ti.Outcome, ti.TestCaseTitle, ti.ErrorMessage));
                }

                foreach (var to in testRun.QueryResultsByOutcome(TestOutcome.Timeout))
                {
                    lstTestRunDetails.Items.Add(string.Format("{0}: {1} - {2}", to.Outcome, to.TestCaseTitle, to.ErrorMessage));
                }

                // Get the test results by user by passing in the Test Foundation Identity
                // testRun.QueryResultsByOwner(TeamFoundationIdentity);
            }

            if(testRuns.Count() == 0)
                lstTestRunDetails.Items.Add("No Test Results have been associated with the selected build");
        }
    }
}

 

6. And the Bonus Power tool…

Thank you for reading this far. If you haven’t already download the Community TFS Build Manager extension from the visual studio extension gallery do so now. The Community TFS Build Manager (TBM) is an open source build management solution developed by the ALM rangers, the build manager simplifies the management of builds in medium to large Team Foundation Server environments. Some common limitations you may run into while dealing with builds through the Visual Studio Build Explorer are not being able to perform bulk operations on builds, create a relationship diagram of the build controller/agents, managing build process templates, etc. The community TFS build manager address that and many other such gaps you may encounter while managing builds through the visual studio build explorer. You can grab the source and read more about the project on CodePlex.

Recently a new feature namely Build Notes has been added to the community TFS Build Manager that let’s you generate an ms word report of the build summary. A sample attached below… Look at the test result section, the trx file is parsed for you to get you a test result list filtered by test result outcome, where failed test also carries the error message. Watch this video or read this post here on what you can achieve with the build notes.

image

 

 image

 

 

Thank you for taking the time out and reading this blog post. If you enjoyed the post, remember to subscribe to http://feeds.feedburner.com/TarunArora.

Check out the other posts on cool tools using TFS API. In love

Posted on Wednesday, May 16, 2012 11:37 PM TFS API | Back to top


Comments on this post: Get Detailed Build Test Results using the TFS API

comments powered by Disqus

Copyright © Tarun Arora [Microsoft MVP] | Powered by: GeeksWithBlogs.net | Join free