Wednesday, January 18, 2012
With support for Windows XP coming to an end on the 8th April 2014 and IE6 standing in so many people’s way of migration, what better time to start taking the leap to a more modern browser?
Microsoft have teamed up with Camwood, Citrix Systems and Quest Software, all application compatibility experts, to help jump-start your move. Join the road shows being organized around the UK between Feb and April to learn more about the technical challenges faced by departments and how you can get around it.
| 
Join us at a date below for a packed agenda discussing the technical challenges faced by IT departments when migrating their users off of IE6. We will establish: · Why you should go through the trouble of migrating; · How you go about identifying those problem applications; · What you can do about them. There are a range of dates to choose from, each offering a slightly different spin and all promising to be thoroughly informative! Take your pick and use the links below to register with our partners. Who, Where & When?
Tues 21st February
Microsoft & Quest Software
Manchester United Football Ground (Old Trafford)
09:30 – 14:00 (+Stadium Tour!)
More Information & Registration
Thurs 22nd March
Microsoft & Citrix Systems (Including newly acquired AppDNA)
Chalfont St. Peter (Easy access via the M40 & M25 as well as rail)
09:00 – 16:00
More Information & Registration
Tues 3rd April
Microsoft & Camwood
London, Cardinal Place
More Information & Registration TBC
Thurs 19th April
Microsoft & Quest Software
Reading Football Ground (Madejski Stadium)
09:30 – 14:00 (+Stadium Tour!)
More Information & Registration On behalf of all the team at Microsoft, we hope to see you there!    
|
Please share the word, after all, 2014 isn’t too far away now 
Saturday, January 07, 2012
Scribe - Test Scribe is a documentation power tool designed to construct documents directly from TFS for test plan and test run artefacts for the purpose of discussion, reporting etc. Scribe only works with Microsoft Test Manager, so if you needed to generate documentation for TFS Builds, what options do you have? You could potentially modify the Build Process template to generate report documentation as the build is executed or you could use the TFS data warehouse to extract data to create rich reports or use the TFS API like I did to create Release Notes.
Recently I stumbled upon Build Manager, build manager is a solution developed by the ALM Rangers, it is intended to ease the management of builds in medium to large Team Foundation Server environments, though it does provide a few features which all users may find useful. You can check out the feature offering in the latest release here. So… I decided to extend Build Manager to create a prototype for Scribe like functionality to generate Build Notes.
Download Source
Download the source and demo executable from here, unzip the contents, you should get the folder structure “..\tfsbuildextensions-72864\TFSBuildManager\MAIN”
- Demo => “..\tfsbuildextensions-72864\TFSBuildManager\MAIN\Demo\TFSBuildManager.Application.exe”
- Source => “..\tfsbuildextensions-72864\TFSBuildManager\MAIN\Source\TFSBuildManager.sln”
Feature Walkthrough
- Open the source code from “..\tfsbuildextensions-72864\TFSBuildManager\MAIN\Source\TFSBuildManager.sln” and set “TFSBuildManager.Application” as the start up project.
- Fire up the build Manager and load builds from across build definitions,
- Select the builds you would like to generate the build notes for | right click and from the context menu choose Build Notes
- Select what you want, mission reduce unnecessary noise!
- I am using Open XML SDK for Microsoft Office to generate the build notes, so even if the client does not have Microsoft Office installed the build notes would be generated. I used the Open XML SDK 2.0 Productivity Tool – which let’s you reflect the code for an existing word document. Though at present I have not spend too much time abstracting the template logic [but this is still a prototype
so I guess that’s acceptable]. I found a very useful walkthrough here that helped me create the build notes template.
And the final few lines of magical code…
public void GenerateBuildNotes(IEnumerable<IBuildDetail> buildDefinitions, BuildNoteOptionsEnum noteOptions)
{
var builds = buildDefinitions.ToArray();
// Pass Values to Template to create the document
var path = String.Format("{0}\\{1}.doc", Path.GetTempPath(), Guid.NewGuid());
Template template = new Template(path, builds, noteOptions, _workItemStore);
template.CreatePackage();
// Open the document
Process.Start(path);
}
- I have bulk tested this for around 125 builds and usually completes in less than a couple of seconds
I need your Help!
Now what I have done so far is a skeleton structure – Build Details, Work Item Details and Change set Details (Download), I want to make build notes comprehensive, but at the same time, the user has the option of enabling/disabling what they want included in the build notes. I need your suggestion on what information would you like to see in the build notes. Some possibilities that come to my mind as I finish this post,
- Build Completing time
- Test Execution Status
- Flag to indicate whether change set is a result of merge
- File details in the change set
You can download the source and have a play around, looking forward to your feedback on further features that can be added to the build notes.
Thank you for taking the time out and reading this blog post. If you enjoyed the post, remember to subscribe to http://feeds.feedburner.com/TarunArora.
Check out the other posts on cool tools using TFS API. 
Tuesday, January 03, 2012
With TFS 2010 a basic installation of TFS has been reduced to a matter of clicks, the pain however lies in getting an appropriate environment provisioned from the Infrastructure team. There will be planned and unplanned downtime as the infrastructure team takes the environment down for patching. There are various TFS hosting services available out there that’ll take this pain point away from you. Some of the leading players amongst others include DiscountASP.net, TeamDevCentral, Praktik Hosting… recently Microsoft has also entered this space with ‘TFS Service’, being part of the Technical Access Preview for TFS Service with Microsoft I had a good play with the offering, you can read more about the TFS Service review here. Though 2 major releases of TFS Service have been successfully rolled out, the offering is still in preview and no official release or pricing information has been made public.
The DiscountAsp.net service is production ready, has received Best TFS Hosting Provider 2011 Award and Brian Harry posted a great recommendation on why you should consider using DiscountAsp.net, these were reasons enough for me to try out DiscountAsp.net Hosted TFS offering myself.
Your enterprise might have a different scale to access the hosting service but personally I would measure against stability, security, easy of use and extensibility.
Availability
Architecturally Team Foundation ‘Server’ 2010 requires the ‘Server’ to be up and available for smooth uninterrupted functioning of development activities, more on server based workspace architecture here. This makes server availability from the hosted service all the more important. DiscountASP.NET promises a planned up time of 100 % per month, any planned maintenance window is communicated and broadcasted at the customer forum well in advance. More on hosted TFS SLA terms & conditions (including details of claiming money back in case SLA is not met).
Security
Ask yourself “Am I comfortable hosting my source code outside my enterprise?” One of the biggest hurdles for customers deciding between on premise and hosted TFS is security. DiscountAsp.Net implements security at all levels,
| Security Standards | Network Security | Data Security | Physical Security |
| - Order forms and control panel use standard SSL encryption
- Enforce strong password usage
- DiscountASP.NET is PCI compliant
- Quarterly PCI scan of all TFS servers
- TrustE EU Safe Harbor certified | - Redundant IP connection
- Redundant Juniper firewalls
- DDOS Migration service subscription
- Inter-server communication on a private subnet
- Only authorized System Administrator can access servers
- System Administrator access is recorded and stored | - Data transfer encrypted with SSL
- File Transfer only allowed using FTP over SSL
- Nightly backups
- Servers and software are kept up to date with security patches. | - SAS70 Certified data center
- RAID 10 disk arrays
- Biometric verification datacenter entry
- 2 level, man-trap, security access
- All camera footage is recorded and held for a minimum of 30 days
- Level 5 bullet-resistant walls / glass (Kevlar Lined walls) & more… |
More details on security and standards offered by DiscountAsp.net for the hosted TFS offering. You might want to ask yourself again whether you are comfortable hosting your source code outside your enterprise!
Ease of Administration
You have the ability to manage users, groups, permissions, add-on, build servers, etc from the web based TFS Hosting Control Panel.
Add-on’s
Some great add-on’s that you can opt for,
- Free migration from Visual Source Safe to Hosted TFS Migration until June 30, 2012 check for details here.
- Urban Turtle – scrum tool for agile project management for a very nominal cost of $5 month/ user.
- TFS Build Server – TFS Build Servers are available from both US based and UK based data centre, more details.
Payment Plans
Very flexible payment plans, you can scale down to just 1 user or scale up to meet the usage demands of your team, expect to pay around $ 20 month/user. With 6+ user enrolment you are promised some excitement with Tiered Pricing Discounts. The contract term is monthly and no minimum contract commitment.
Check the most up to date pricing information here.
Resources
DiscountAsp.net has great a ecosystem of community support forums, KB, tutorials and other useful resources to get you on your feet up and running in the shortest possible time. Last but not the least you don’t want to invest in an offering or company that is here today and might disappear tomorrow. DiscountAsp.net has a great track record established in 2002 the company has grown tremendously, the team has constantly added one niche offering after the other, the latest being managed TFS Hosting, this shows great commitment and confidence in the brand.
Conclusion
I have 5 team projects on my hosted TFS collection and am using TFS to its full ALM potential, the service is very stable and there has been no downtime reported in my run of 2 months so far. The centralised website based administration of TFS 2010 makes TFS administration seamless. I have been able to easily scale up and down the team size every time I needed to without any bottleneck delays. Very impressed by DiscountAsp.Net so far. If you need more information click here, you can also follow @Discountasp for up to date news & offers.
Sunday, January 01, 2012
January 2012 will go down as one of the most memorable months! What a fantastic way to start the year, received an email from Microsoft telling me that I have been presented with the Microsoft MVP Award in the Visual Studio Application Lifecycle Management Competency for 2012.
Dear Tarun Arora,
Congratulations! We are pleased to present you with the 2012 Microsoft® MVP Award! This award is given to exceptional technical community leaders who actively share their high quality, real world expertise with others. We appreciate your outstanding contributions in Visual Studio ALM technical communities during the past year.
Also in this email:
• About your MVP Award Gift
• How to claim your award benefits
• Your MVP Identification Number
• MVP Award Program Code of Conduct
The Microsoft MVP Award provides us the unique opportunity to celebrate and honour your significant contributions and say "Thank you for your technical leadership."
Nestor Portillo
Director
Community & Online Support
My Recipe for MVP!
The secret is there is no one single recipe for making an MVP! As for me… over the previous years I have been actively contributing to the Microsoft Technical Forums, sharing solutions on codeplex & other communities and writing about my experiences on my blog. Microsoft pays a lot of attention to contributions made in the ecosystem by individuals and makes every effort to acknowledge their expertise and contributions.
Charles Sterling who sits on the other side of the table has a great blog post on the process for selecting an MVP.
All I can say is, be committed and be honest in your contributions!
After all, Microsoft could not be everywhere so they decided to create MVP’s…
Thank you Microsoft for acknowledging the efforts, among others I would really like to thank Brian Keller, Anu Bharadwaj & Charles Sterling for their continued support and direction! I am delighted to have become part of this prestigious community, I will try my best and more to live up to the expectations!
Wish you a very Happy New Year 2012!
Best Wishes, Tarun.
Saturday, December 24, 2011
Agile taught us to release early and release often, Agile also taught us to get the end user involved more frequently and directly. Goes without saying that release notes are an important part of any release. But what you don’t want to be doing is manually putting together the release notes with each frequent release you do. TFS lets you manage the application lifecycle with great ease & minimal administrative overhead, if your team is already checking in the code against work items and if you are using the TFS build infrastructure, good news is you already have all the raw material for your release notes stored in the TFS repository, in this blog post I’ll show you how to use the TFS API to generate the release notes from the Team builds.
So… How does it work?
- => Click connect to get all Build Definitions for a Team Project
- => Filter by excluding Disabled Build Definitions, or select the no.
- of builds to return per build definition, or exclude failed,
- partially successful, stopped and in progress builds.
- => Click ‘Generate Release Notes’ to generate the release
- notes, you can also export the results to PDF
Let’s get started, follow the steps below or download a working solution & an executable from here.
1. What you need…
To be able to extract the work item details from the build, you need to ensure,
- Developers check in code against work items – Download and Install TFS Power tools and enable check in policy to enforce check ins against work items.
- Enable the association of work items with change sets – If you are using the default template that ships with TFS 2010, edit the build definition and under the Process tab in the Advanced settings you will notice the option ‘Associate Change sets and Work Items’ set this to true.
2. TFS API – Programmatically get Work Items, Change sets, etc. from Team Builds
- 1. Let’s start by connecting to TFS programmatically using the API
private TfsTeamProjectCollection _tfs;
private string _selectedTeamProject;
private IBuildServer _bs;
private WorkItemStore _wis;
// Connect to TFS and pick team project
private void ConnectToTfsAndPickAProject()
{
TeamProjectPicker tfsPP = new TeamProjectPicker(TeamProjectPickerMode.SingleProject, false);
tfsPP.ShowDialog();
this._tfs = tfsPP.SelectedTeamProjectCollection;
this._selectedTeamProject = tfsPP.SelectedProjects[0].Name;
}
- 2. Get all Build Definitions for the selected Team Project programmatically
// Get All Build Definitions for the selected Team Project
private IBuildDefinition[] GetAllBuildDefinitionsFromTheTeamProject()
{
_bs = _tfs.GetService<IBuildServer>();
return _bs.QueryBuildDefinitions(_selectedTeamProject);
}
- 3. Get All Builds in the build definition for specific build Definition specifications programmatically, this helps you narrow down the search and get the selected few filtered results, like in the snippet below, we specify the maximum number of builds to return, what build quality builds to return, etc.
// Get All Builds for the selected build definition
private void cmbBuilds_SelectedIndexChanged(object sender, EventArgs e)
{
var def = (((ComboBox)sender).SelectedItem) as IBuildDefinition;
var def = _bs.CreateBuildDetailSpec(_selectedTeamProject);
int noRequeted; // set to 1 in case no value passed
def.MaxBuildsPerDefinition = (int.TryParse(txtNoOfBuilds.Text.ToString(), out noRequeted)) ? noRequeted : 1;
def.QueryOrder = BuildQueryOrder.FinishTimeDescending;
def.DefinitionSpec.Name = def.Name;
def.Status = chkOnlySuccessful.Checked ? BuildStatus.Succeeded : BuildStatus.All;
var builds = _bs.QueryBuilds(def).Builds;
lstBuilds.Items.Clear();
foreach (var build in builds)
{
lstBuilds.Items.Add(build);
lstBuilds.DisplayMember = "BuildNumber";
}
lstBuilds.SelectionMode = SelectionMode.MultiExtended;
}
- 4. Get Associated Work Items for the selected build. You can also get the Associated Change sets, Build errors, Build Warnings, Work flow details, Changeset Ids, Configuration summaries, Opened work item ids, etc, check for more details for InformationNodeConverters. Notice in the snippet below that I am excluding a work item if it has already been added to the release notes as part of the previous selected build.
// Data I want to display on the report
public class ReleaseNote
{
public string BuildNumber { get; set; }
public int WorkItemId { get; set; }
public string WorkItemTitle { get; set; }
public string QcRef { get; set; }
}
// Get the work item details off the selected Builds
private void btnReleaseNotes_Click(object sender, EventArgs e)
{
var workItemIds = new List<int>();
_wis = _tfs.GetService<WorkItemStore>();
var workItems = new List<WorkItem>();
var releaseNotes = new List<ReleaseNote>();
foreach (IBuildDetail build in ((ListBox)lstBuilds).SelectedItems)
{
List<IWorkItemSummary> wis = InformationNodeConverters.GetAssociatedWorkItems(build);
foreach (var wi in wis)
{
var w = _wis.GetWorkItem(wi.WorkItemId);
if (w.Type != null && w.Type.Name == "Defect")
{
if (!releaseNotes.Select(r => r.WorkItemId).Contains(w.Id))
releaseNotes.Add(new ReleaseNote()
{
BuildNumber = build.BuildNumber,
WorkItemId = w.Id,
WorkItemTitle = w.Title,
QcRef = w["QC Reference"].ToString()
});
}
}
}
ReleaseNoteBindingSource.DataSource = releaseNotes;
this.reportViewer1.RefreshReport();
}
Next… Create a Report using the Report Wizard and associate it to the Release Note Binding Source. Download a working solution and executable from here.
3. Review & Next Steps…
You can use the same logic with in a custom activity to churn out the release note details in the format you want in the build output, or use the code in a web application , host it in your network and easily let the team query the release notes as and when they need to, or host this in the cloud so the vendors outside the network can easily access the release notes as well, you could also convert this in to a VS extension, right click the build definition and you have the option to generate the release notes. You can also extend the logic further and include more functionality. Happy to hear your feedback! If you have any recommendations or questions, feel free to leave a comment. Thank you for taking the time out and reading this blog post. If you enjoyed the post, remember to subscribe to http://feeds.feedburner.com/TarunArora.
Check out the other posts on cool tools using TFS API.
Wednesday, December 14, 2011
Welcome to the hidden treasure of “MTM.config.exe”
Navigate to “C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE” and open the 
1. Can I change the number of Action Recording steps selected when a Bug is created?
When you are exploratory testing using Microsoft Test Manager or performing a feedback session using Feedback Manager and decide to raise a bug or create a test case, the last 4*10 Action recordings are selected by default. You can change the number of action recordings selected by default by changing the value of the key DefaultNumberOfActions, the multiplier of 4 of this value is the number of steps selected by default.
2. How Does MTM Know Win + E = Explorer, Can I Change this?
While recording a coded UI Test if you press Win + E the windows explorer window gets opened, similarly when you press Win + R the run prompt gets opened and this gets recorded as part of your test. This can be good or bad depending on whether you would like this global key combination to be part of your Coded UI test. Good news is that Microsoft Test Manager is highly configurable,
In the app settings section you will see the “GlobalHotKeys” section, this contains a list of all the hot key combinations MTM will honour during the coded UI test recording session, removing Windows, E; or Windows, R; from here will remove the resolution of this combination to Windows Explorer and Run Prompt respectively.
You also have the option of changing the key combination for HoverKey or simply turn off the recording for hovering action by setting the value of RecordImplicitHover to false. You also have the option of changing the Think time threshold value.
3. Where are the Short cut Key combinations stored And…. Can i change them?
The short cut key combinations for Passing Test Step, Failing Test Step, Moving to Next or Previous Test Step and Capturing Image are stored in the MTM.exe.config file.
Can I use a different tool than MsPaint to annotate the screen captures I take?
The Answer is yes, change set the key AnnotateImage to true, my personal preference is Paint.Net, this setting can be changed in the MTM.exe.config by changing the value of the key AnnotationTool
<add key="AnnotationTool" value="%ProgramFiles%\Paint.Net\PaintDotNet.exe"/>
Test Run and Lab Settings,
The MTM.config.exe also contains configuration settings for Lab and Test Run pertaining to environment refresh and data collection timespans that can also be changed from here.
4. Oh no something is wrong with MTM, how do I troubleshoot?
When i was working on trying to get the exploratory session information using the TFS API, I was trying to figure out what queries MTM fires in the background when I load the ‘View Exploratory Test Sessions’ page, I turned on fiddler in the background but to my despair I could not trap any communication. By default Soap Tracing is disabled in Microsoft Test Manager, which is fair considering it has a performance overhead.
To enable soap tracing in MTM, you will need to edit the MTM.exe.config and set the value of TestManagement.EnableSoapTracing to true,
<add key="TestManagement.EnableSoapTracing" value="true" />
Now set the trace level to between 1 – 4, for diagnostic level tracing set the level to 4,
<system.diagnostics>
<switches>
<!-- Trace settings for MTR -->
<!-- Change the value to change the level of tracing you want.
You must use integral values for "value".
0 ==> off,
1 ==> error,
2 ==> warn,
3 ==> info,
4 ==> verbose.
Note that each value is inclusive of the last.
-->
<add name="UITestTraceLevel" value="0" />
<add name="EqtTraceLevel" value="0" />
<add name="TestManagement" value="4" />
<add name="TeamBuild" value="0" />
<add name="TeamFoundationSoapProxy" value="0" />
</switches>
<trace autoflush="true"></trace>
</system.diagnostics>
You can use ‘Fiddler’ or download ‘DebugView’ a free trace listener to listen in!
Apart from that you can also enable Test Execution Event Log Tracing by adding the following key to the config
<!-- EnableTestExecutionEventLogTracing: Errors that occur during execution of tests will be written to the event log. The default is "yes"; override with "no". -->
<add key="EnableTestExecutionEventLogTracing" value="yes"/>
<!-- Log folder. Trace logs for background recorder and playback are placed in this folder. -->
<add key="UITestTraceFolder" value="%TEMP%\UITestLogs"/>
<!-- RemotingTimeoutSeconds: Maximum time a remoting call can last in seconds (default 10 minutes). -->
<add key="RemotingTimeoutSeconds" value="600"/>
Watch this video for more details on troubleshooting using Tracing…
Troubleshooting using Tracing
What techniques do you use to trouble shoot when you run in to an issue with TFS, Build or MTM?
Thank you for taking the time out and reading this blog post. If you enjoyed the post, remember to subscribe to http://feeds.feedburner.com/TarunArora.
Monday, December 12, 2011
In this blog post I’ll show you how to use the TFS 2011 API to get the Exploratory Testing & Feedback Manager session information programmatically.
Introduction
TFS 2011 includes Exploratory Testing and Feedback Manager, the first empowers the testers to explore the application and second empowers the stake holders to explore the application. This rhymes a bit… that’s because… under the hood both use the same infrastructure to drive and store the session information. If you try to run both at the same time on a machine you will run in to the following warning message.
Note both Feedback manager and Exploratory testing store the results in the table [Tfs_DefaultCollection].[dbo].[tbl_Session] in the TFS operational database.
Note - Any direct interaction (including select statements) against the TFS operational databases is strongly discouraged by the Product Team. All such interactions should be done through the TFS SDK.
How to get the Exploratory Session and Feedback Manager Session programmatically using TFS API…
Download Working Demo… Or follow the steps below,
1. Add references
using Microsoft.TeamFoundation.Client;
using Microsoft.TeamFoundation.TestManagement.Client;
2. Connect to TFS and select a Team Project
private TfsTeamProjectCollection _tfs;
private string _selectedTeamProject;
private void ConnectToTfsAndPickAProject()
{
// Connect to TFS and pick team project
TeamProjectPicker tfsPP = new TeamProjectPicker(TeamProjectPickerMode.SingleProject, false);
tfsPP.ShowDialog();
this._tfs = tfsPP.SelectedTeamProjectCollection;
this._selectedTeamProject = tfsPP.SelectedProjects[0].Name;
}
3. Get access to Session Helper
// Get the Session Helper
var tms = _tfs.GetService<ITestManagementService>();
var sessionHelper = tms.GetTeamProject(_selectedTeamProject).SessionHelper;
4. Now use the session helper to query for the session Information, for example in the snippet below I am simply getting all sessions,
// Get Session by Query
var sessionByQuery = sessionHelper.Query("Select * from Session");
dataGridView1.DataSource = sessionByQuery.ToList();
dataGridView1.AutoGenerateColumns = true;
// I could write a more specific query
var sessionQueryExample = sessionHelper.Query("Select * from Session where TestPlanId =3");
BONUS – Go one step further...
5. Get Session Information by Link Id: In the Links section in a Bug you can see the test results attachment, you can use the ‘FindByLinkId’ method to get the details of that session programmatically.
// Get Session details for a bug raised during session testing
// 1. Get the bug
var qBug = @"SELECT
[System.Id],
[System.Links.LinkType],
[System.WorkItemType],
[System.Title],
[System.State],
[Custom.TestingTour],
[Microsoft.VSTS.Common.Priority]
FROM WorkItems
WHERE [System.TeamProject] = '@project'
AND [System.WorkItemType] = 'Bug'"
.Replace("@project", _selectedTeamProject);
// Get the Wi Store service
var wis = _tfs.GetService<WorkItemStore>();
var queryBug = new Query(wis, qBug);
var bugs = queryBug.RunQuery();
// 2. Get the Uri of the session from the bug
foreach (WorkItem bug in bugs)
{
foreach (Link l in bug.Links)
{
if (l.ArtifactLinkType != null && l.ArtifactLinkType.Name == "Test Result")
{
// 3. Get Session Information By URI
var sessionByUri = sessionHelper.FindByLink(new Uri(((ExternalLink)l).LinkedArtifactUri));
}
}
}
Download the working demo.
You might also enjoy reading => How to customize Process Template In TFS 2011.
Hope you enjoyed this post! Remember to subscribe to http://feeds.feedburner.com/TarunArora.
Enjoy 
Sunday, December 11, 2011
In this blog post I’ll be walking you through the process of customizing the TFS 2011 Scrum 2.0 Process Template to add a new work item type, introducing new work item queries, modifying existing queries, adding new reports, etc. To give you a bit of context, at the 2011 ALM Summit Anu Bharadwaj presented a talk on Exploratory Testing highlighting how the Team @ Microsoft took advantage of MTM 2011 to test various VS ALM products in the “fun” way by organizing “Testing Tours.” To dive deep in to the subject I decided to modify the scrum 2.0 process template by introducing a ‘Testing Tour’ type work item and setting up some reporting around it to monitor, track and report on the testing inroads made through Testing tours.
1. What is a testing Tour?
This question has greatly been answered already. An excerpt from the post here…
“Tours are essentially used to define a theme for exploration. If you were a tourist in a new city, how would you explore the city? Perhaps you would visit the famous landmarks in a quick tour, perhaps you would do a detailed walking tour of just one of your favourite sites, perhaps you would visit the lesser known attractions and avoid the touristy places. Likewise, if you were a tester testing a new app, how would you test it? Perhaps you would do a quick sales demo walking through the primary scenarios, perhaps you would test one scenario in great detail, perhaps you would test some of the lesser advertised features. Any theme that you choose to apply to your exploration is informed by the learning you have had from your previous explorations. You know your developer is lousy at exception handling - a tour like "Intelligent tourist tour" (where the tourist asks hard questions to the guide :D) would help - you could try all the hard boundary value inputs or rarer scenarios bound to produce a boundary condition. You know the app has performance issues when more than a few users hit the server - you could try the "Carnival tour" where your entire test team logs on to the client altogether and parties on it! :)
My favourite tour is the Back Alley tour where you exercise some of the lesser known paths in the application, the Fedex tour where you can track a data structure from point to point to see how it gets transmuted, the All Nighter tour can be helpful in catching some nasty memory leaks”…
2. Making sure we’re on the right track…
The Goal is to customise the scrum 2.0 process template such that we have new work item type to track all ‘Testing Tours’ that are performed in a sprint. The Testing tour work item will be child to the Product Back log Item. It should be possible to identify when what testing tour was conducted, whether the tour is impeded, pending, in progress or complete. The tester responsible for the tour should be able to see the testing tour work item in his/her to-do view in visual studio team explorer. As part of the bigger picture the testing team should be able to run daily stand ups using the testing tours reports which give a representation of testing tour run frequency, bug count by testing tours & testing tour coverage by stories.
Order of changes required to reach the finish line,
I. Download existing Scrum 2.0 process Template
II. Adding New Work Item Type Definition
III. Adding/Editing Work Item Queries
IV. Uploading the new Process Template and Creating a new Team Project
V. Testing the to-do view, work item links & queries in Visual Studio
VI. Verifying Changes to the TFS data warehouse
VII. Writing new SSRS Reports
VIII. Writing new Reports using TFS API (covered in Part II)
IX. Wearing the tester hat to start touring! (covered in Part II)
In case you are looking for a short cut, you can download the modified process template from here.
3. Customising Microsoft Scrum 2.0 Process Template
To try this out you will need TFS 2011 preview, Visual Studio 2011 preview & Microsoft Test Manager 11 preview. If you don’t already have a play pit with all this set up, consider downloading a ready to use TFS 2011 ALM preview VM from Brian Keller’s Blog. Let’s get started…
3.1 – Download Scrum 2.0 process Template
You can download the Microsoft Scrum 2.0 Preview Process Template from the Process Template Manager from Visual Studio 2011 Preview Team Explorer.
Open Visual Studio | Go to Team Explorer Home | Click Settings | Click Process Template Manager | From the Process Template Manager modal window click Download
3.2 - Adding New Work Item Type Definition
Navigate to the path you downloaded the process template. You will see a lot of subfolders, basically a process template is a collection of files that together define various process elements of a team project in Team Foundation Server. The key folders for us will be WorkItem Tracking and Reports,
Navigate to the ..\WorkItem Tracking\TypeDefinitions folder, the xml files this folder contains are the definitions of the work item types. Create a copy of ‘Task.xml’ and rename the copy to ‘TestingTour.xml’.
Note – If you have Visual Studio 2010 available on a machine, you can cheat by opening ‘TestingTour.xml’ in the Process Template Editor. This will not only make editing the TestingTour.xml easy but also allow you to preview the changes.
1. Specify the work item name and description, make sure the name here matches the name of the file.
2. Add a new custom field ‘Testing Tour’, make it reportable and build a list of all the Testing tours you would like to show up in the testing tour drop down list.
3. Modify the Activity Field and set ‘Testing’ as the default value, so that when the Testing Tour work item is created the Activity Field list which contains ‘Development’, ‘Design’, ‘Documentation’, etc is set to ‘Testing’.
4. Now from the Layout tab in the ‘Details’ Group add a new control and change the Label to ‘Testing Tour:’ and set the Field Name to ‘Custom.TestingTour’. After making the changes you can click on the Preview Form button to see a preview of what the work item ‘TestingTour’ will look like.
Note – If you are having trouble or do not have Visual Studio 2010 then consider downloading the final ‘TestingTour.xml’ from here.
5. Navigate to the ..\WorkItem Tracking folder and open WorkItems.xml, the newly created ‘TestingTour.xml’ will needed to be added to the WorkItems.xml in order for the new work item type to be created during the team project creation. In the <WORKITEMTYPES> tag add the path to the TestingTour.xml as shown in the screen shot below,
With what we have done so far, if you were to create a new team project from this process template you will be able to see the new ‘Testing Tour’ work item type. Screen shot below…
3.3 - Adding/Editing Work Item Queries
Navigate to the ..\WorkItem Tracking\Queries folder, the wiq files contain the work item queries that are created for you when the team project is created.
We will be adding two new work item queries,
- ‘MyTestingTours.wiq’ – To see all the testing tours assigned to me. The below query will also show the Testing Tour the work item will be used to cover.
<?xml version="1.0" encoding="utf-8"?>
<WorkItemQuery Version="1">
<Wiql>
SELECT [System.Id],
[System.Title],
[System.State],
[System.AssignedTo],
[Custom.TestingTour]
FROM WorkItems
WHERE [System.TeamProject] = @project
AND [System.WorkItemType] = 'TestingTour'
AND [System.AssignedTo] = @me
ORDER BY [System.Id]
</Wiql>
</WorkItemQuery>
- ‘TestingTours.wiq’ – To all testing tours by PBI in the current sprint. This will be a linked query and will show the parent work item i.e. the requirement work item linked to the Testing Tour work item. This will help us report on the product backlog coverage by Testing tour.
<?xml version="1.0" encoding="utf-8"?>
<WorkItemQuery Version="1">
<Wiql>
SELECT
[System.Id],
[System.Links.LinkType],
[System.WorkItemType],
[System.Title],
[System.State],
[Custom.TestingTour],
[Microsoft.VSTS.Common.Priority]
FROM WorkItemLinks
WHERE ([Source].[System.TeamProject] = @project
AND [Source].[System.IterationPath] UNDER '$$PROJECTNAME$$\Release 1\Sprint 1'
AND [Source].[System.WorkItemType] = 'TestingTour')
And ([System.Links.LinkType] <> '')
And ([Target].[System.WorkItemType] <> '')
ORDER BY [Microsoft.VSTS.Common.Priority] mode(MayContain)
</Wiql>
</WorkItemQuery>
Along with this we’ll be editing,
- ‘BlockedTasks.wiq’ – To account for any impeded testing tours in a sprint that have purposely not been removed.
<?xml version="1.0" encoding="utf-8"?>
<WorkItemQuery Version="1">
<Wiql>
SELECT [System.Id],
[System.WorkItemType],
[System.Title],
[Microsoft.VSTS.Common.BacklogPriority],
[System.AssignedTo],
[System.State],
[Microsoft.VSTS.CMMI.Blocked]
FROM WorkItems
WHERE [System.TeamProject] = @project
AND [System.IterationPath] UNDER '$$PROJECTNAME$$\Release 1\Sprint 1'
AND ( [System.WorkItemType] = 'Task' OR [System.WorkItemType] = 'TestingTour' )
AND [Microsoft.VSTS.CMMI.Blocked] = 'Yes'
AND [System.State] <> 'Removed'
ORDER BY [Microsoft.VSTS.Common.BacklogPriority]
</Wiql>
</WorkItemQuery>
- ‘MyWork-ToDo.wiq’ – To add testing tours to the work item hub view in Visual Studio Team Explorer. Now the interesting thing is that this query picks up the Work Items that are assigned to me not from the work item type directly but from the ‘My Work Category’ group that is defined in the ..\WorkItem Tracking\Categories.xml which I’ll show you how to modify in the next step.
<?xml version="1.0" encoding="utf-8"?>
<WorkItemQuery Version="1">
<Wiql>
SELECT
[System.Id],
[System.WorkItemType],
[System.Title],
[System.AssignedTo],
[System.State],
[Microsoft.VSTS.Common.BacklogPriority]
FROM WorkItems
WHERE [System.TeamProject] = @project
AND [System.AssignedTo] = @me
AND ([System.State] <> 'Done')
AND ([System.State] <> 'Removed')
AND [System.WorkItemType] in group 'My Work Category'
ORDER BY [Microsoft.VSTS.Common.BacklogPriority], [System.Id]
</Wiql>
</WorkItemQuery>
- ‘SprintBacklog.wiq’ – To account for testing tours along with tasks, bugs & PBI’s in a sprint. As you can see below, if the Testing Tour work item is linked to PBI then it will only be picked up if it has not been removed from the sprint.
<?xml version="1.0" encoding="utf-8"?>
<WorkItemQuery Version="1">
<Wiql>
SELECT [System.Id],
[System.Title],
[Microsoft.VSTS.Common.BacklogPriority],
[System.AssignedTo],
[System.State],
[Microsoft.VSTS.Scheduling.RemainingWork],
[Microsoft.VSTS.CMMI.Blocked],
[System.WorkItemType]
FROM WorkItemLinks
WHERE ([Source].[System.TeamProject] = @project
AND [Source].[System.IterationPath] UNDER '$$PROJECTNAME$$\Release 1\Sprint 1'
AND ( [Source].[System.WorkItemType] = 'Product Backlog Item' OR [Source].[System.WorkItemType] = 'Bug'
OR [Source].[System.WorkItemType] = 'TestingTour' OR [Source].[System.WorkItemType] = 'Task' )
AND [Source].[System.State] <> 'Removed')
And ([System.Links.LinkType] = 'System.LinkTypes.Hierarchy-Forward')
And (( [Target].[System.WorkItemType] = 'Task' OR [Target].[System.WorkItemType] = 'TestingTour' )
AND [Target].[System.State] <> 'Removed')
ORDER BY [Microsoft.VSTS.Common.BacklogPriority],
[Microsoft.VSTS.Scheduling.Effort],
[Microsoft.VSTS.Scheduling.RemainingWork]
mode(Recursive)
</Wiql>
</WorkItemQuery>
- ‘UnfinishedWork.wiq’ – To track unfinished work on the testing tours in a sprint, this query differs from the above only in the work item state comparison.
<?xml version="1.0" encoding="utf-8"?>
<WorkItemQuery Version="1">
<Wiql>
SELECT [System.Id],
[System.Title],
[Microsoft.VSTS.Common.BacklogPriority],
[System.AssignedTo],
[System.State],
[Microsoft.VSTS.Scheduling.RemainingWork],
[Microsoft.VSTS.CMMI.Blocked],
[System.WorkItemType]
FROM WorkItemLinks
WHERE ([Source].[System.TeamProject] = @project
AND [Source].[System.IterationPath] UNDER '$$PROJECTNAME$$\Release 1\Sprint 1'
AND ( [Source].[System.WorkItemType] = 'Product Backlog Item' OR [Source].[System.WorkItemType] = 'Bug'
OR [Source].[System.WorkItemType] = 'TestingTour' OR [Source].[System.WorkItemType] = 'Task' )
AND [Source].[System.State] <> 'Removed'
AND [Source].[System.State] <> 'Done') And ([System.Links.LinkType] = 'System.LinkTypes.Hierarchy-Forward')
And (( [Target].[System.WorkItemType] = 'Task' OR [Target].[System.WorkItemType] = 'TestingTour' )
AND [Target].[System.State] <> 'Done'
AND [Target].[System.State] <> 'Removed')
ORDER BY [Microsoft.VSTS.Common.BacklogPriority],
[Microsoft.VSTS.Scheduling.Effort],
[Microsoft.VSTS.Scheduling.RemainingWork]
mode(Recursive)
</Wiql>
</WorkItemQuery>
- ‘WorkInProgress.wiq’ – To track the in progress tours in a sprint along this also brings back the remaining work for the completion of the tour as part of the query result.
<?xml version="1.0" encoding="utf-8"?>
<WorkItemQuery Version="1">
<Wiql>
SELECT [System.Id],
[System.WorkItemType],
[System.Title],
[System.AssignedTo],
[System.State],
[Microsoft.VSTS.Scheduling.RemainingWork]
FROM WorkItems
WHERE [System.TeamProject] = @project
AND [System.IterationPath] UNDER '$$PROJECTNAME$$\Release 1\Sprint 1'
AND ( [System.WorkItemType] = 'Task' OR [System.WorkItemType] = 'TestingTour' )
AND [System.State] = 'In Progress'
ORDER BY [System.AssignedTo],
[Microsoft.VSTS.Common.BacklogPriority]
</Wiql>
</WorkItemQuery>
As I mentioned earlier, the ‘My Work Category’ group that is defined in the ..\WorkItem Tracking\Categories.xml will need to be modified to include the ‘TestingTour’ type work item.
Also, the new queries that we have defined will not be picked up until we explicitly set this up in the ..\WorkItem Tracking\WorkItems.xml. In the Current Sprint QueryFolder add the ‘TestingTour.wiq’ and in the My Work QueryFolder add the MyTestingTours.wiq.
3.4 - Uploading the new Process Template and Creating a new Team Project
Finishing touches, before you upload the process template, remember to rename the process template by modifying ..\Microsoft Visual Studio Scrum 2.0 – Preview 1\ProcessTemplate.xml failing to do this may result in you overwriting the existing process template during upload.
As a best practice, I would highly recommend modifying the ..\Microsoft Visual Studio Scrum 2.0 – Preview 1\Classification\Classification.xml to include a property for process template Name, you will understand the joy this brings when you try and modify the process template programmatically using the TFS API, more on this here.
If you are still reading, congratulations, you have done all the hard work, it’s time to create a new Team Project using the process template you have successfully customized for ‘Testing Tours’.
1. Open Process Template Manager (Refer to 3.1) and click on Upload and navigate to the customized process template and click ok.
2. From the Team Project Selector Control choose ‘New Team Project’ or even better, type ‘New Team Project’ in the Quick Launch control
3. Let the wizard lead you, make sure you select the customized process template you uploaded in the previous steps.
Great! So the project ‘Testing Tour – Demo’ has been created using the customized Scrum Process Template. To try out the queries it is important that you create at least 1 Product Back Log Item, 1 Task associated to that Product Back Log Item and 2 Testing Tours (1 associated to the Product Backlog Item and 1 orphaned). Make sure you assign all these work items to the Iteration ‘Testing Tour – Demo\Release1\Sprint1’.
To add the Testing Tour work item to the Product Backlog Item, you can go to Links, click 
to add a new TestingTour type work item.
Create the other Testing Tour work Item from the Team Explorer Work Item Page and do not associate it to the Product Backlog Item, assign it to Sprint 1 and set the field Blocked to ‘Yes’.
Now that we have some requirements, tasks and testing tours set up, let’s move to the next section to test the customizations done to the process template to track, monitor and manage testing tours.
3.5 - Testing the to-do view, work item links & queries in Visual Studio
Open Visual Studio | Team Explorer | Work Items | Shared Queries
- From under the
folder run the 
query. We can see all work items, including testing tours, both forward associated to the PBI and the orphaned testing tour work item.
- From under the
folder run the 
query. We can see that the blocked testing tour work item shows up in the query result. This makes tracking any impeded tours very easily to track and manage.
- From under the
folder run the 
query. You can see the parent work item i.e. PBI associated to the testing tour. Makes tracking tour by requirement so easy to track. You can also see the Testing tour type as part of the query results.
- From under the
folder run the 
or the 
query. You should be able to track all testing tours a long with the other work item types that have unfinished work or are in progress.
- From under the
folder run the 
query. You will be able to see all testing tours assigned to you.
- From under the
folder run the 
query. You will be able to see all tasks, bugs, pbi’s and testing tours (‘My Work Category’ group) that are currently assigned to you.
As a tester if you were to use Visual Studio for developing let’s say coded UI or performance tests you could take benefit of ‘My Work’, ‘Suspend’ & ‘Code Review’ feature as well using the Testing Tour work item.
All in all a custom work item for Testing Tours can simplify monitoring & tracking of tours! Let’s move on to the next section to see how changes made in the process template impact the TFS 2011 data warehouse.
3.6 - Verifying Changes to the TFS data warehouse
When the ‘Testing Tour’ field was created in section 3.2 we set Reporting to ‘Dimension’. This means Custom.TestingTour should have been added to the TFS Data warehouse.
Open SSMS or SQL Express and connect to the TFS_Warehouse, if you are using the Preview VM then use ‘VSALM’ as the server name, if you are not sure, open the Team Foundation Server Administration console and check the connection string to identify the db server name.
- Select from [Tfs_Warehouse].[dbo].[_WitFields] and verify you can see the Testing Tour field added to this table.
- Now that the Field has been added to the work item you can take advantage of reporting using tfs data warehouse, you can see the field Testing Tour has also been added to all the Work Item Views. For Example, Select from the [Tfs_Warehouse].[dbo].[CurrentWorkItemView] and you will see the column Custom_TestingTour
Note – In case you can’t see any values populated for the Custom_TestingTour validate that the TFS Data warehouse has processed at least once after you created the new work items in section 3.4. If you are having trouble with TFS Data warehouse consider going through the helpful guidance here.
3.7 - Writing new SSRS Reports
I will be covering reporting on Testing Tours in detail in the next part, but to get you started, I’ll show you a sample – Get All Testing Tours by Iteration for a specific Team Project. Get All details such as the current state of the test tour, effort remaining, last state, user, iteration start date, end date, etc.
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
-- =============================================
-- Author: Tarun Arora
-- Create date: 10 December 2011
-- Description: Get All Test Tours
-- By Iteration & Team Project
-- =============================================
CREATE PROCEDURE usp.TestTours.TestToursByProjectAndIteration
@project nvarchar(50), @Iteration nvarchar(max)
AS
BEGIN
SET NOCOUNT ON;
-- Uncomment for Testing
-- Declare @Iteration nvarchar(max),
-- @Project nvarchar(50)
-- Set @Project = (Select ProjectNodeGUID from dbo.DimTeamProject
-- where ProjectNodeName = 'Testing Tour - Demo' and IsDeleted = 0)
-- Set @Iteration = (select IterationGUID from dbo.DimIteration
-- where ProjectGUID = @Project and IterationPath = '\Testing Tour - Demo\Release 1\Sprint 1')
Select
wi.[LastUpdatedDateTime]
,[Microsoft_VSTS_Scheduling_RemainingWork]
,[PreviousState]
,[System_State]
,[System_ChangedDate]
,[System_Id]
,[System_Title]
,[System_ChangedBy]
,[System_Reason]
,[System_AssignedTo]
,[System_WorkItemType]
,[System_CreatedDate]
,[System_CreatedBy]
,[Microsoft_VSTS_Common_Activity]
,[Microsoft_VSTS_CMMI_Blocked]
,[Custom_TestingTour]
,[ProjectNodeName]
,[AreaName]
,wi.[IterationPath]
,wi.[IterationName]
,wi.[IterationGUID]
,i.[StartDate]
,i.[FinishDate]
FROM [Tfs_Warehouse].[dbo].[CurrentWorkItemView] wi
INNER JOIN [Tfs_Warehouse].[dbo].[DimIteration] i
ON i.IterationGUID = wi.IterationGUID
WHERE [System_WorkItemType] = 'TestingTour'
AND [ProjectNodeGUID] = @Project
And wi.[IterationGUID] = @Iteration
END
GO
By passing the Iteration path, query the DimIteration to get the IterationGuid, Iteration Start and Finish Dates. Join with the CurrentWorkItemView and get all test tour details. The execution of the above query gets the results below,
Simple & Sweet! Since the data is in the warehouse, you can take full advantage of the extensive reporting options that come with TFS. It is also possible to connect the warehouse using an excel and put together quick pivots to measure the impact test tours are having on your overall ALM process.
4. Review and What’s next?
In Part-1 of ‘Customizing TFS 2011 Scrum process template to introduce Test Tours to start exploratory testing using MTM11’ overall we explored,
- How to set up a new work item type
- Make the new work item type reportable
- Add/Update Work Item Queries
- Query the TFS Data warehouse for the new work item type.
- Download a copy of the customized process template.
This brings up close to the finish line… But not quite there yet! 
In Part 2, I’ll be diving deep to build some firm reports using both TFS Data warehouse and TFS API to effectively represent the testing tour run frequency, bug count by testing tours & testing tour coverage by stories. As part of the bigger picture the testing team should be able to run daily stand ups using the testing tour reports. If you are looking for some inspiration around using TFS API – check out this post on MTM Testing Scorecard using TFS API.
Hope you enjoyed this post! Remember to subscribe to http://feeds.feedburner.com/TarunArora. I would love to hear your feedback, if you have any recommendations on things that I should consider or any questions or feedback, feel free to leave a comment.
See you in Part II.
Saturday, December 03, 2011
Is this blog post worth your time?
It’s not challenging setting up the Load Test Rig in the Cloud, it’s more challenging to convince enterprises to embrace the cloud. Not all enterprises will accept their business critical applications being profiled from a VM hosted outside their network. Though through Windows Azure Connect you can join the Cloud hosted Role to the on premise domain effectively forcing the domain policies on the external VM but how many enterprises will authorize you to do this? I have worked with various Energy Trading companies who admire the flexibility that Azure brings but often shy away from the implementation because they still consider it risky. There is no immediate solution to this, but I sure think that the thought process could be transformed by implementing the change gradually. The client may not be willing to experiment with the business critical applications, but you stand a good chance if you can identify the not so important applications and highlight the benefits of Azure-ing the Test Rig such as,
- Zero Administration overhead.
- Since Microsoft manages the hardware (patching, upgrades and so forth), as well as application server uptime, the involvement of IT pros is minimized.
- Near to 100% guaranteed uptime.
- Utilizing the power of Azure compute to run a heavy virtual user load.
- Benefiting from the Azure flexibility, destroy Test Agents when not in use, takes < 25 minutes to spin up a new Test Agent.
- Most important test Network Latency, (network latency and speed of connection are two different things – usually network latency is very hard to test), by placing the Test Agents in Microsoft Data centres around the globe, one can actually test the lag in transferring the bytes not because of a slow connection but because the page has been requested from the other side of the globe.
Welcome to Part 3, In Part 1 we discussed the advantages of taking the Test Rig to the cloud, In Part 2 we manually set up the Test Rig in the cloud, In Part 3 I intent to completely automate the end to end deployment of the Test Agent to the cloud.
Another Azure Gem – Startup Tasks in Worker Role
Let’s get started by what we need to automate the Test Agent deployment to the cloud. We can use startup tasks to perform operations before the role starts. Operations that one might want to perform include installing a component, registering COM components, setting registry keys, or starting a long running process. Read more about start up tasks here. I will be leveraging startup tasks to install Visual Studio Test Agent software and further registering the Test Agent to the controller.
I. Packaging everything in to the Solution
In Part 2 we created a Solution using the Windows Azure Project Template, I’ll be extending the same solution,
DOWNLOAD A WORKING SOLUTION OR follow the instructions below,
1. StartupTask - Create a new Folder under the WorkerRole1 Project and call it StartupTask.
2. VS2010Agents.zip - If you haven’t already, download the Visual Studio Agents Software from MSDN, I used the ‘en_visual_studio_agents_2010_x86_x64_dvd_509679.iso’ extracted the iso and zipped the TestAgents folder and called it VS2010Agents.zip. Add the VS2010Agents.zip to the folder StartupTask, the approximate size of the VS2010Agents.zip would be around 75 MB.
3. Setup.cmd - Create a new file Setup.cmd and add it to the StartupTask folder. Setup.cmd will just be the workflow manager. The cmd file will call the Setup.ps1 script that we will generate in the next step and log the output to a log file. Your cmd file should look like below.
REM Allow Power Shell scripts to run unrestricted
powershell -command "set-executionpolicy Unrestricted"
REM Unzip VS2010 Agents
powershell -command ".\setup.ps1" -NonInteractive > out.txt
4. Setup.ps1 - Unzipping VS2010Agents.zip – I found a very helpful post that shows you how to unzip a file using powershell. Create a function Unzip-File that accepts two parameters, basically, the source and destination, verifies that the source location is valid creates the destination location and unzips the file to the destination location. So, your Setup.ps1 file should look like below.
function Unzip-File {
PARAM (
$zipfilename="E:\AppRoot\StartupTask\VS2010Agents.zip",
$destination="C:\resources\temp\vsagentsetup"
)
if(test-path($zipfilename))
{
$shellApplication = new-object -com shell.application
$zipPackage = $shellApplication.NameSpace($zipfilename)
$destinationFolder = $shellApplication.NameSpace($destination)
$destinationFolder.CopyHere($zipPackage.Items(), 1040)
}
else
{
echo "The source ZIP file does not exist";
}
}
md "C:\resources\temp\vsagentsetup"
Unzip-File "E:\AppRoot\StartupTask\VS2010Agents.zip" "C:\resources\temp\vsagentsetup"
5. SetupAgent.ini – Unattended Installation - When the setup.ps1 has been run, it would unzip the VisualStudioAgents.zip to C:\resources\temp\vsagentsetup. Now you need to install the Agents software, but you would want to run an unattended Installation. I found a great walkthrough on MSDN that shows you how to create an unattended installation of Visual Studio, I have used this walkthrough to create the setupAgent.ini file, don’t worry if you are having trouble creating the ini file, the entire solution is available for download at the bottom of the post.
Once you have created or downloaded the setupagent.ini file, you need to append instructions to the setup.cmd to start the unattended install using the setupagent.ini that you just created. You can add the below statements to the setup.cmd.
REM Unatteneded Install of Test Agent Software
C:\resources\temp\vsagentsetup\VS2010TestAgents\testagent\setup.exe /q /f /norestart /unattendfile e:\approot\StartupTask\setupagent.ini
6. ConfigAgent.cmd – Now you need to configure the agent to work with the test controller. This is easy to do, you can call the TestAgentConfig.exe from the command line by passing the user details and controller details. The MSDN post here was helpful. Create a new file call it ConfigAgent.cmd, it should look like below,
"D:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\TestAgentConfig.exe" configureAsService ^
/userName:vstestagent /password:P2ssw0rd /testController:TFSSPDEMO:6901
you will need to append instructions to setup.cmd to call the ConfigAgent.cmd and run it as a scheduled task, with elevated permissions. You can add the below statements to the setup.cmd
REM Create a task that will run with full network privileges.
net start Schedule
schtasks /CREATE /TN "Configure Test Agent Service" /SC ONCE /SD 01/01/2020 ^
/ST 00:00:00 /RL HIGHEST /RU admin /RP P2ssw0rd /TR e:\approot\configagent.cmd /F
schtasks /RUN /TN "Configure Test Agent Service"
7. Setupfw.cmd – You will need to open up ports for communication between the test controller and the test agent. In normal developer environment at times you can get away by turning off the firewall, however, this will not be a possibility in a corporate network. Found a very useful post that shows you how to configure the Windows 2008 Server Advanced firewall from the CLI. Once you are through the Setupfw.cmd should look like below,
netsh advfirewall firewall add rule name="RPC" ^
dir=in action=allow service=any enable=yes profile=any localport=rpc protocol=tcp
netsh advfirewall firewall add rule name="RPC-EP" ^
dir=in action=allow service=any enable=yes profile=any localport=rpc-epmap protocol=tcp
.
.
netsh advfirewall firewall add rule name="Port 6910 TCP" ^
dir=in action=allow service=any enable=yes profile=any localport=6910 protocol=tcp
netsh advfirewall firewall add rule name="Port 6910 UDP" ^
dir=in action=allow service=any enable=yes profile=any localport=6910 protocol=udp
II. How will the Startup Tasks be executed?
- Set the value of Copy to Output Directory to Copy Always. Failing to do so will result in a deployment error. This step will ensure that all the startupTask files are added to the package and uploaded to the Test Agent VM. By default the files are uploaded to E:\AppRoot\
- Open ServiceDefinition.csdef and include a trigger to call the workflow cmd to start the execution of the command file. The csdef file should look like below once you have added the Startup Task tags.
<?xml version="1.0" encoding="utf-8"?>
<ServiceDefinition name="WindowsAzureProject2"
xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceDefinition">
<WorkerRole name="WorkerRole1" vmsize="Small">
<Imports>
<Import moduleName="Diagnostics" />
<Import moduleName="RemoteAccess" />
<Import moduleName="RemoteForwarder" />
<Import moduleName="Connect" />
</Imports>
<Startup>
<Task commandLine="StartupTask\setup.cmd" executionContext="elevated" taskType="background" />
<Task commandLine="StartupTask\setupfw.cmd" executionContext="elevated" taskType="background" />
</Startup>
<LocalResources>
</LocalResources>
</WorkerRole>
</ServiceDefinition>
- ExecutionContext – Make sure you set the execution context to run as elevated. The execution of the cmd file will fail if the file is executed in non elevated mode.
- TaskType – You have the option to set the task type as simple, backgroup, foreground. If you set the taskType to simple, then until the startup task is complete the deployment of the VM will not complete. I won’t recommend running the startup Task as simple, you should run it as background, in case something goes wrong you will have the ability to remotely login to the machine and investigate the reason for the failure. a
FAQ: Something has failed but i don’t know what?
Well to be honest there isn’t too much that can go wrong during the deployment, but in case there is a failure, your best bet is to look at the VM creation logs, the output.log created at the time of running the cmd file. I found some great troubleshooting tips and tricks and gotchas at the blog posts (this, this) of the channel 9 @cloudcover team.
III. Publish, Scaling up or down
Now the most important step, once you have finished writing all the startup tasks and changing the setting of the startup tasks to copy always, you are ready to test the deployment.
You have two options to carry out the deployment,
- Publish From Visual Studio – Right click the Azure project and choose Publish
- Package From Visual Studio – Once the package is ready, you can start deployment from the Windows Azure Portal.
Once the deployment is complete you still have a handle on changing the configurations. For example, in the screen shot below, I can change the Instance count configuration from 1 to 10. This will trigger an additional deployment of 9 more instances using the same package that was used to create the original machine. Benefit, you can scale up your environment in a very short interval. One also has the option to stop or delete the machines once they are not required.
Review and What’s next?
In this blog post, we have automated the process of spinning up the Test Agents in windows Azure. You can download the demo solution from here.
You can read the earlier posts in the series here,
Earlier Azure-ing your Test Rig posts
- Part 1 – Load Testing in the Cloud
- Part 2 – Load Testing in the Cloud
I have various blog posts on Performance Testing with Visual Studio Ultimate, you can follow the links and videos below,
Blog Posts:
- Part 1 – Performance Testing using Visual Studio 2010 Ultimate
- Part 2 – Performance Testing using Visual Studio 2010 Ultimate
- Part 3 – Performance Testing using Visual Studio 2010 Ultimate
Videos:
- Test Tools Configuration & Settings in Visual Studio
- Why & How to Record Web Performance Tests in Visual Studio Ultimate
- Goal Driven Load Testing using Visual Studio Ultimate
I have some ideas around Azure and TFS that I am continuing to work on, not giving away any hints here, but remember to subscribe to http://feeds.feedburner.com/TarunArora I’ll be blogging the results of these experiments soon! Hope you enjoyed this post, I would love to hear your feedback! If you have any recommendations on things that I should consider or any questions or feedback, feel free to leave a comment.
Sunday, November 27, 2011
Welcome to Part 2, In Part 1 we discussed the advantages of creating a Test Rig in the cloud, the Azure edge and the Test Rig Topology we want to get to. In Part 2, Let’s start by understanding the components of Azure we’ll be making use of followed by manually putting them together to create the test rig, so… let’s get down dirty start setting up the Test Rig.
What Components of Azure will I be using for building the Test Rig in the Cloud?
To run the Test Agents we’ll make use of Windows Azure Compute and to enable communication between Test Controller and Test Agents we’ll make use of Windows Azure Connect.
Azure Connect
The Test Controller is on premise and the Test Agents are in the cloud (How will they talk?). To enable communication between the two, we’ll make use of Windows Azure Connect. With Windows Azure Connect, you can use a simple user interface to configure IPsec protected connections between computers or virtual machines (VMs) in your organization’s network, and roles running in Windows Azure. With this you can now join Windows Azure role instances to your domain, so that you can use your existing methods for domain authentication, name resolution, or other domain-wide maintenance actions. For more details refer to an overview of Windows Azure connect. A very useful video explaining everything you wanted to know about Windows Azure connect.
Azure Compute
Windows Azure compute provides developers a platform to host and manage applications in Microsoft’s data centres across the globe. A Windows Azure application is built from one or more components called ‘roles.’ Roles come in three different types: Web role, Worker role, and Virtual Machine (VM) role, we’ll be using the Worker role to set up the Test Agents. A very nice blog post discussing the difference between the 3 role types.
Developers are free to use the .NET framework or other software that runs on Windows with the Worker role or Web role. Developers can also create applications using languages such as PHP and Java. More on Windows Azure Compute.
Each Windows Azure compute instance represents a virtual server...
| Virtual Machine Size |
CPU Cores |
Memory |
Cost Per Hour |
| Extra Small |
Shared |
768 MB |
$0.04 |
| Small |
1 |
1.75 GB |
$0.12 |
| Medium |
2 |
3.50 GB |
$0.24 |
| Large |
4 |
7.00 GB |
$0.48 |
| Extra Large |
8 |
14.00 GB |
$0.96 |
You might want to review the Windows Azure Pricing FAQ.
Let’s Get Started building the Test Rig…
Configuration
| Machine |
Role |
Comments |
| VM – 1 |
Domain Controller for Playpit.com |
On Premise |
| VM – 2 |
TFS, Test Controller |
On Premise |
| VM – 3 |
Test Agent |
Cloud |
In this blog post I would assume that you have the domain, Team Foundation Server and Test Controller Installed and set up already. If not, please refer to the TFS 2010 Installation Guide and this walkthrough on MSDN to set up your Test Controller. You can also download a preconfigured TFS 2010 VM from Brian Keller's blog, Brian also has some great hands on Labs on TFS 2010 that you may want to explore.
I. Lets start building VM – 3: The Test Agent
-
Download
the Windows Azure SDK and Tools
- Open Visual Studio and create a new Windows Azure Project using the Cloud Template
Choose the Worker Role for reasons explained in the earlier post
- The WorkerRole.cs implements the Run() and OnStart() methods, no code changes required. You should be able to compile the project and run it in the compute emulator (The compute emulator should have been installed as part of the Windows Azure Toolkit) on your local machine.
- We will only be making changes to WindowsAzureProject, open ServiceDefinition.csdef. Ensure that the vmsize is small (remember the cost chart above). Import the “Connect” module. I am importing the Connect module because I need to join the Worker role VM to the Playpit domain.
<?
xml
version
="1.0" encoding="utf-8"?>
<ServiceDefinition name="WindowsAzureProject2"
xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceDefinition">
<WorkerRole name="WorkerRole1" vmsize="Small">
<Imports>
<Import moduleName="Diagnostics" />
<Import moduleName="Connect"/>
</Imports>
</WorkerRole>
</ServiceDefinition>
- Go to the ServiceConfiguration.Cloud.cscfg and note that settings with key ‘Microsoft.WindowsAzure.Plugins.Connect.%%%%’ have been added to the configuration file. This is because you decided to import the connect module. See the config below.
<?
xml
version
="1.0" encoding="utf-8"?>
<ServiceConfiguration serviceName="WindowsAzureProject2"
xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceConfiguration" osFamily="1"osVersion="*">
<Role name="WorkerRole1">
<Instances count="1"/>
<ConfigurationSettings>
<Setting name="Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString" value="UseDevelopmentStorage=true" />
<Setting name="Microsoft.WindowsAzure.Plugins.Connect.ActivationToken" value="" />
<Setting name="Microsoft.WindowsAzure.Plugins.Connect.Refresh" value="" />
<Setting name="Microsoft.WindowsAzure.Plugins.Connect.WaitForConnectivity" value="" />
<Setting name="Microsoft.WindowsAzure.Plugins.Connect.Upgrade" value="" />
<Setting name="Microsoft.WindowsAzure.Plugins.Connect.EnableDomainJoin" value="" />
<Setting name="Microsoft.WindowsAzure.Plugins.Connect.DomainFQDN" value="" />
<Setting name="Microsoft.WindowsAzure.Plugins.Connect.DomainControllerFQDN" value="" />
<Setting name="Microsoft.WindowsAzure.Plugins.Connect.DomainAccountName" value="" />
<Setting name="Microsoft.WindowsAzure.Plugins.Connect.DomainPassword" value="" />
<Setting name="Microsoft.WindowsAzure.Plugins.Connect.DomainOU" value="" />
<Setting name="Microsoft.WindowsAzure.Plugins.Connect.Administrators" value="" />
<Setting name="Microsoft.WindowsAzure.Plugins.Connect.DomainSiteName" value="" />
</ConfigurationSettings>
</Role>
</ServiceConfiguration>
Let’s go step by step and understand all the highlighted parameters and where you can find the values for them.
- osFamily – By default this is set to 1 (Windows Server 2008 SP2). Change this to 2 if you want the Windows Server 2008 R2 operating system. The Advantage of using osFamily = “2” is that you get Powershell 2.0 rather than Powershell 1.0. In Powershell 2.0 you could simply use “powershell -ExecutionPolicy Unrestricted ./myscript.ps1” and it will work while in Powershell 1.0 you will have to change the registry key by including the following in your command file “reg add HKLM\Software\Microsoft\PowerShell\1\ShellIds\Microsoft.PowerShell /v ExecutionPolicy /d Unrestricted /f” before you can execute any power shell. The other reason you might want to move to os2 is if you wanted IIS 7.5.
- Activation Token – To enable communication between the on premise machine and the Windows Azure Worker role VM both need to have the same token. Log on to Windows Azure Management Portal, click on Connect, click on Get Activation Token, this should give you the activation token, copy the activation token to the clipboard and paste it in the configuration file. Note – Later in the blog I’ll be showing you how to install connect on the on premise machine.
- EnableDomainJoin – Set the value to true, ofcourse we want to join the on windows azure worker role VM to the domain.
- DomainFQDN, DomainControllerFQDN, DomainAccountName, DomainPassword, DomainOU, Administrators – This information is specific to your domain. I have extracted this information from the ‘service manager’ and ‘Active Directory Users and Computers’. Also, i created a new Domain-OU namely ‘CloudInstances’ so all my cloud instances joined to my domain show up here, this is optional. You can encrypt the DomainPassword – refer to the instructions here. Or hold fire, I’ll be covering that when i come to certificates and encryption in the coming section.
Now once you have filled all this information up, the configuration file should look something like below,
<?
xml
version
="1.0" encoding="utf-8"?>
<ServiceConfiguration serviceName="WindowsAzureProject2"
xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceConfiguration" osFamily="2" osVersion="*">
<Role name="WorkerRole1">
<Instances count="1" />
<ConfigurationSettings>
<Setting name="Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString" value="UseDevelopmentStorage=true" />
<Setting name="Microsoft.WindowsAzure.Plugins.Connect.ActivationToken" value="45f55fea-f194-4fbc-b36e-25604faac784" />
<Setting name="Microsoft.WindowsAzure.Plugins.Connect.Refresh" value="" />
<Setting name="Microsoft.WindowsAzure.Plugins.Connect.WaitForConnectivity" value="" />
<Setting name="Microsoft.WindowsAzure.Plugins.Connect.Upgrade" value="" />
<Setting name="Microsoft.WindowsAzure.Plugins.Connect.EnableDomainJoin" value="true" />
<Setting name="Microsoft.WindowsAzure.Plugins.Connect.DomainFQDN" value="play.pit.com" />
<Setting name="Microsoft.WindowsAzure.Plugins.Connect.DomainControllerFQDN" value="WIN-KUDQMQFGQOL.play.pit.com" />
<Setting name="Microsoft.WindowsAzure.Plugins.Connect.DomainAccountName" value="playpit\Administrator" />
<Setting name="Microsoft.WindowsAzure.Plugins.Connect.DomainPassword" value="************************" />
<Setting name="Microsoft.WindowsAzure.Plugins.Connect.DomainOU" value="OU=CloudInstances, DC=Play, DC=Pit, DC=com" />
<Setting name="Microsoft.WindowsAzure.Plugins.Connect.Administrators" value="Playpit\Administrator" />
<Setting name="Microsoft.WindowsAzure.Plugins.Connect.DomainSiteName" value="" />
</ConfigurationSettings>
</Role>
</ServiceConfiguration>
- Next we will be enabling the Remote Desktop module in to the ServiceDefinition.csdef, we could make changes manually or allow a beautiful wizard to help us make changes. I prefer the second option. So right click on the Windows Azure project and choose Publish
- Now once you get the publish wizard, if you haven’t already you would be asked to import your Windows Azure subscription, this is simply the Msdn subscription activation key xml. Once you have done click Next to go to the Settings page and check ‘Enable Remote Desktop for all roles’.
- As soon as you do that you get another pop up asking you the details for the user that you would be logging in with (make sure you enter a reasonable expiry date, you do not want the user account to expire today). Notice the more information tag at the bottom, click that to get access to the certificate section. See screen shot below.
- From the drop down select the option to create a new certificate
- In the pop up window enter the friendly name for your certificate. In my case I entered ‘WAC – Test Rig’ and click ok. This will create a new certificate for you. Click on the view button to see the certificate details. Do you see the Thumbprint, this is the value that will go in the config file (very important). Now click on the Copy to File button to copy the certificate, we will need to import the certificate to the windows Azure Management portal later. So, make sure you save it a safe location.
- Click Finish and enter details of the user you would like to create with permissions for remote desktop access, once you have entered the details on the ‘Remote desktop configuration’ screen click on Ok. From the Publish Windows Azure Wizard screen press Cancel. Cancel because we don’t want to publish the role just yet and Yes because we want to save all the changes in the config file.
- Now if you go to the ServiceDefinition.csdef file you will see that the RemoteAccess and RemoteForwarder roles have been imported for you.
<?
xml
version
="1.0" encoding="utf-8"?>
<ServiceDefinition name="WindowsAzureProject2"
xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceDefinition">
<WorkerRole name="WorkerRole1" vmsize="Small">
<Imports>
<Import moduleName="Diagnostics" />
<Import moduleName="Connect" />
<Import moduleName="RemoteAccess" />
<Import moduleName="RemoteForwarder"/>
</Imports>
</WorkerRole>
</ServiceDefinition>
- Now go to the ServiceConfiguration.Cloud.cscfg file and you see a whole bunch for setting “Microsoft.WindowsAzure.Plugins.RemoteAccess.%%%” values added for you.
<?
xml
version
="1.0" encoding="utf-8"?>
<ServiceConfiguration serviceName="WindowsAzureProject2"
xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceConfiguration" osFamily="2" osVersion="*">
<Role name="WorkerRole1">
<Instances count="1" />
<ConfigurationSettings>
<Setting name="Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString" value="UseDevelopmentStorage=true" />
<Setting name="Microsoft.WindowsAzure.Plugins.Connect.ActivationToken" value="45f55fea-f194-4fbc-b36e-25604faac784" />
<Setting name="Microsoft.WindowsAzure.Plugins.Connect.Refresh" value="" />
<Setting name="Microsoft.WindowsAzure.Plugins.Connect.WaitForConnectivity" value="" />
<Setting name="Microsoft.WindowsAzure.Plugins.Connect.Upgrade" value="" />
<Setting name="Microsoft.WindowsAzure.Plugins.Connect.EnableDomainJoin" value="true" />
<Setting name="Microsoft.WindowsAzure.Plugins.Connect.DomainFQDN" value="play.pit.com" />
<Setting name="Microsoft.WindowsAzure.Plugins.Connect.DomainControllerFQDN" value="WIN-KUDQMQFGQOL.play.pit.com" />
<Setting name="Microsoft.WindowsAzure.Plugins.Connect.DomainAccountName" value="playpit\Administrator" />
<Setting name="Microsoft.WindowsAzure.Plugins.Connect.DomainPassword" value="************************" />
<Setting name="Microsoft.WindowsAzure.Plugins.Connect.DomainOU" value="OU=CloudInstances, DC=Play, DC=Pit, DC=com" />
<Setting name="Microsoft.WindowsAzure.Plugins.Connect.Administrators" value="Playpit\Administrator" />
<Setting name="Microsoft.WindowsAzure.Plugins.Connect.DomainSiteName" value="" />
<Setting name="Microsoft.WindowsAzure.Plugins.RemoteAccess.Enabled" value="true" />
<Setting name="Microsoft.WindowsAzure.Plugins.RemoteAccess.AccountUsername" value="Administrator" />
<Setting name="Microsoft.WindowsAzure.Plugins.RemoteAccess.AccountEncryptedPassword"
value="MIIBnQYJKoZIhvcNAQcDoIIBjjCCAYoCAQAxggFOMIIBSgIBADAyMB4xHDAaBgNVBAMME1dpbmRvd
3MgQXp1cmUgVG9vbHMCEGa+B46voeO5T305N7TSG9QwDQYJKoZIhvcNAQEBBQAEggEABg4ol5Xol66Ip6QKLbAPWdmD4ae
ADZ7aKj6fg4D+ATr0DXBllZHG5Umwf+84Sj2nsPeCyrg3ZDQuxrfhSbdnJwuChKV6ukXdGjX0hlowJu/4dfH4jTJC7sBWS
AKaEFU7CxvqYEAL1Hf9VPL5fW6HZVmq1z+qmm4ecGKSTOJ20Fptb463wcXgR8CWGa+1w9xqJ7UmmfGeGeCHQ4QGW0IDSBU6ccg
vzF2ug8/FY60K1vrWaCYOhKkxD3YBs8U9X/kOB0yQm2Git0d5tFlIPCBT2AC57bgsAYncXfHvPesI0qs7VZyghk8LVa9g5IqaM
Cp6cQ7rmY/dLsKBMkDcdBHuCTAzBgkqhkiG9w0BBwEwFAYIKoZIhvcNAwcECDRVifSXbA43gBApNrp40L1VTVZ1iGag+3O1" />
<Setting name="Microsoft.WindowsAzure.Plugins.RemoteAccess.AccountExpiration" value="2012-11-27T23:59:59.0000000+00:00" />
<Setting name="Microsoft.WindowsAzure.Plugins.RemoteForwarder.Enabled" value="true" />
</ConfigurationSettings>
<Certificates>
<Certificatename="Microsoft.WindowsAzure.Plugins.RemoteAccess.PasswordEncryption"
thumbprint="AA23016CF0BDFC344400B5B82706B608B92E4217"thumbprintAlgorithm="sha1" />
</Certificates>
</Role>
</ServiceConfiguration>
Okay let’s look at them one at a time,
- Enabled - Yes, we would like to enable Remote Access.
- AccountUserName – This is the user name you entered while you were on the publish windows azure role screen, as detailed above.
- AccountEncrytedPassword – Try and decode that, the certificate is used to encrypt the password you specified for the user account. Remember earlier i said, either use the instructions or wait and i’ll be showing you encryption, now the user account i am using for rdp has the same password as my domain password, so i can simply copy the value of the AccountEncryptedPassword to the DomainPassword as well.
- AccountExpiration – This is the expiration as you specified in the wizard earlier, make sure your account does not expire today.
- Remote Forwarder – Check out the documentation, below is how I understand it,
-- One role in an application that implements a remote desktop connection must import the RemoteForwarder module. The two modules work together to enable the remote desktop connections to role instances.
-- If you have multiple roles defined in the service model, it does not matter which role you add the RemoteForwarder module to, but you must add it to only one of the role definitions.
- Certificate – Remember the certificate thumbprint from the wizard, the on premise machine and windows azure role machine that need to speak to each other must have the same thumbprint. More on that when we install Windows Azure connect Endpoints on the on premise machine.
- As i said earlier, in this blog post, I’ll be showing you the manual process so i won’t be scripting any star up tasks to install the test agent or register the test agent with the TFS Server. I’ll be showing you all this cool stuff in the next blog post, that’s because it’s important to understand the manual side of it, it becomes easier for you to troubleshoot in case something fails.
Having said that, the changes we have made are sufficient to spin up the Windows Azure Worker Role aka Test Agent VM, have it connected with the play.pit.com domain and have remote access enabled on it. Before we deploy the Test Agent VM we need to set up Windows Azure Connect on the TFS Server.
II. Windows Azure Connect: Setting up Connect on VM – 2 i.e. TFS & Test Controller
Glad you made it so far, now to enable communication between the on premise TFS/Test Controller and Azure-ed Test Agent we need to enable communication. We have set up the Azure connect module in the Test Agent configuration, now the connect end points need to be enabled on the on premise machines, let’s have a look at how we can do this.
- Log on to VM – 2 running the TFS Server and Test Controller
- Log on to the Windows Azure Management Portal and click on Virtual Network
- Click on Virtual Network, if you already have a subscription you should see the below screen shot, if not, you would be asked to complete the subscription first
- Click on Install Local Endpoints from the top left on the panel and you get a url appended with a token id in it, remember the token i showed you earlier, in theory the token you get here should match the token you added to the Test Agent config file.
- Copy the url to the clip board and paste it in IE explorer (important, the installation at present only works out of IE and you need to have cookies enabled in order to complete the installation). As stated in the pop up, you can NOT download and run the software later, you need to run it as is, since it contains a token. Once the installation completes you should see the Windows Azure connect icon in the system tray.
Right click the Azure Connect icon, choose Diagnostics and refer to this link for diagnostic detail terminology.
-
NOTE – Unfortunately I could not see the Windows Azure connect icon in the system tray, a bit of binging with Google revealed that the azure connect icon is only shown when the ‘Windows Azure Connect Endpoint’ Service is started. So go to services.msc and make sure that the service is started, if not start it, unfortunately again, the service did not start for me on a manual start and i realised that one of the dependant services was disabled, you can look at the service dependencies and start them and then start windows azure connect. Bottom line, you need to start Windows Azure connect service before you can proceed. Please refer here on MSDN for more on Troubleshooting Windows Azure connect. (Follow the next step as well)
- Now go back to the Windows Azure Management Portal and from Groups and Roles create a new group, lets call it ‘Test Rig’. Make sure you add the VM – 2 (the TFS Server VM where you just installed the endpoint).
- Now if you go back to the Azure Connect icon in the system tray and click ‘Refresh Policy’ you will notice that the disconnected status of the icon should change to ready for connection.
III. Importing Certificate in to Windows Azure Management Portal
But before that you need to import the certificate you created in Step I in to the Windows Azure Management Portal.
- Log on to the Windows Azure Management Portal and click on ‘Hosted Services, Storage Accounts & CDN’ and then ‘Management Certificates’ followed by Add Certificates as shown in the screen shot below
- Browse to the location where you saved the certificate earlier, remember… Refer to Step I in case you forgot.
- Now you should be able to see the imported certificate here, make sure the thumbprint of the certificate matches the one you inserted in the config files
IV. Publish Windows Azure Worker Role aka Test Agent
Having completed I, II and III, you are ready to publish the Test Agent VM – 3 to the cloud.
- Go to Visual Studio and right click the Windows Azure project and select Publish. Verify the infomration in the wizard, from the advanced settings tab, you can also enabled capture of intellitrace or profiling information.
- Click Next and Click Publish! From the view menu bar select the Windows Azure Activity Log window.
- Now you should be able to see the deployment progress in real time.
In the Windows Azure Management Portal, you should also be able to see the progress of creation of a new Worker Role.
- Once the deployment is complete you should be able to RDP (go to run prompt type mstsc and in the pop up the machine name) in to the Test Agent Worker Role VM from the Playpit network using the domain admin user account. In case you are unable to log in to the Test Agent using the domain admin user account it means the process of joining the Test Agent to the domain has failed! But the good news is, because you imported the connect module, you can connect to the Test Agent machine using Windows Azure Management Portal and troubleshoot the reason for failure, you will be able to log in with the user name and password you specified in the config file for the keys ‘RemoteAccess.AccountUsername, RemoteAccess.EncryptedPassword (just that enter the password unencrypted)’, fix it or manually join the machine to the domain. Once you have managed to Join the Test Agent VM to the Domain move to the next step.
- So, log in to the Test Agent Worker Role VM with the Playpit Domain Administrator and verify that you can log in, the machine is connected to the domain and the connect service is successfully running. If yes, give your self a pat on the back, you are 80% mission accomplished!
- Go to the Windows Azure Management Portal and click on Virtual Network, click on Groups and Roles and click on Test Rig, click Edit Group, the edit the Test Rig group you created earlier. In the Connect to section, click on Add to select the worker role you have just deployed. Also, check the ‘Allow connections between endpoints in the group’ with this you will enable to communication between test controller and test agents and test agents/test agents. Click Save.
Now, you are ready to deploy the Test Agent software on the Worker Role Test Agent VM and configure it to work with the Test Controller.
V. Configuring VM – 3: Installing Test Agent and Associating Test Agent to Controller
Log in to the Worker Role Test Agent VM that you have just successfully deployed, make sure you log in with the domain administrator account.
- Download the All Agents software from MSDN, ‘en_visual_studio_agents_2010_x86_x64_dvd_509679.iso’, extract the iso and navigate to where you have extracted the iso. In my case, i have extracted the iso to “C:\Resources\Temp\VsAgentSetup”. Open the Test Agent folder and double click on setup.exe. Once you have installed the Test Agent you should reach the configuration window. If you face any issues installing TFS Test Agent on the VM, refer to the walkthrough on MSDN.
- Once you have successfully installed the Test Agent software you will need to configure the test agent. Right click the test agent configuration tool and run as a different user. i.e. an Administrator. This is really to run the configuration wizard with elevated privileges (you might have UAC block something's otherwise).
- In the run options, you can select ‘service’ you do not need to run the agent as interactive un less you are running coded UI tests. I have specified the domain administrator to connect to the TFS Test Controller. In real life, i would never do that, i would create a separate test user service account for this purpose. But for the blog post, we are using the most powerful user so that any policies or restrictions don’t block you.
- Click the Apply Settings button and you should be all green! If not, the summary usually gives helpful error messages that you can resolve and proceed. As per my experience, you may run in to either a permission or a firewall blocking communication issue.
- And now the moment of truth! Go to VM –2 open up Visual Studio and from the Test Menu select Manage Test Controller
- Mission Accomplished! You should be able to see the Test Agent that you have just configured here,
VI. Creating and Running Load Tests on your brand new Azure-ed Test Rig
I have various blog posts on Performance Testing with Visual Studio Ultimate, you can follow the links and videos below,
Blog Posts:
- Part 1 – Performance Testing using Visual Studio 2010 Ultimate
- Part 2 – Performance Testing using Visual Studio 2010 Ultimate
- Part 3 – Performance Testing using Visual Studio 2010 Ultimate
Videos:
- Test Tools Configuration & Settings in Visual Studio
- Why & How to Record Web Performance Tests in Visual Studio Ultimate
- Goal Driven Load Testing using Visual Studio Ultimate
Now that you have created your load tests, there is one last change you need to make before you can run the tests on your Azure Test Rig, create a new Test settings file, and change the Test Execution method to ‘Remote Execution’ and select the test controller you have configured the Worker Role Test Agent against in our case VM – 2
So, go on, fire off a test run and see the results of the test being executed on the Azur-ed Test Rig. 
Review and What’s next?
A quick recap of the benefits of running the Test Rig in the cloud and what i will be covering in the next blog post AND I would love to hear your feedback!
Advantages
- Utilizing the power of Azure compute to run a heavy virtual user load.
- Benefiting from the Azure flexibility, destroy Test Agents when not in use, takes < 25 minutes to spin up a new Test Agent.
- Most important test Network Latency, (network latency and speed of connection are two different things – usually network latency is very hard to test), by placing the Test Agents in Microsoft Data centres around the globe, one can actually test the lag in transferring the bytes not because of a slow connection but because the page has been requested from the other side of the globe.
Next Steps
The process of spinning up the Test Agents in windows Azure is not 100% automated. I am working on the Worker process and power shell scripts to make the role deployment, unattended install of test agent software and registration of the test agent to the test controller automated. In the next blog post I will show you how to make the complete process unattended and automated.
Remember to subscribe to http://feeds.feedburner.com/TarunArora. Hope you enjoyed this post, I would love to hear your feedback! If you have any recommendations on things that I should consider or any questions or feedback, feel free to leave a comment.
See you in Part III
.