Wednesday, April 25, 2012
As many of you probably have noticed by now, Visual Studio Database Projects are not supported in the next version of Visual Studio (currently named
Visual Studio 11 Beta). When you open a solution containing a VSDB project, VS11 wants to convert it to a SQL Server Developer Tools project instead.
This project type ships with SQL Server and has a feature set that covers most of the functionality of the VSDB project, plus some new features, such
a support for SQL 2012 and SQL Azure. A feature comparison list between the two project types can be found here:
http://blogs.msdn.com/b/ssdt/archive/2011/11/21/sql-server-data-tools-ctp4-vs-vs2010-database-projects.aspx
Once you have converted your project to a SSDT project, you will find that most of the functionality is very similar to VSDB, how you work with
schema objects, schema comparisons etc. Deploying a SSDT project is called Publish and is available in the Visual Studio context menu:
When you invoke the Publish command, Visual Studio will launch the Publish Profile dialog, where you can configure how and where you want to
deploy the database:
There are lots of options that you can configure, and these options are often different depending on the target environment. For example, locally you
typically want to recreate the database every time you deploy, but when deploying to a test server, you probably only want to update it incrementally
without removing any existing data. The settings that you enter can be stored in a separate profile file, which you will use when you are deploying the database.
So, create a publish profile for each environment that you want to deploy to. In the following example, I have one profile for deploying to my local
machine, and in addition publish profiles for the test and production environments:
(Note that you can right-click a publish profile and mark it as default. This is the profile that will be chosen when you select Publish in Visual Studio, so
in this case I would select Local.publish.xml)
The Publish command calls the Publish MSBuild target which will eventually call the SqlPublishTask MSBuild task which will do the work of deploying your
database. This means that the deployment of the database project is easy to integrate into TFS Build, since you can just specify that you want to invoke
the Publish target as part of your build:
Here, I have chosen to deploy the database using the Test profile, which would typically by a remote server used for testing of the build.
Using SQLCMD variables
Sometimes you need to use parameters in your scripts, e.g. values that you can pass in dynamically when the script is executed. These are called
SQLCMD variables, and you can define these on the properties page of the database project:
Here I have defined a variable called $(TargetServer), and given it a default value of localhost. Then I have references this variable inside a post
deployment script in side the project, like this:
EXEC master..xp_cmdshell 'bcp Daatabase.[dbo].[Table] in "TableContent.dat" -T -c -S$(TargetServer)'
This is a scenario we had at a client recently, where they used the BCP utility to bulk insert lots of data into a few tables as part of the deployment.
To be able to run BCP against different target servers (dev, test etc) in my build, I used the SQLCMD variable.
When you publish your database from Visual Studio, it will prompt you to give the variables a value. But when deploying from a build, the value need
to be set per configuration. This is done by opening the publish profile file for the target environment and store that value there:
Select “Save Profile As” and save it as your target publish profile. Since we are specifying our publish profile in our build definition, it will populate
the variables with the correct values.
Sunday, April 01, 2012
Today I received an email from Microsoft stating that:
Dear Jakob Ehn,
Congratulations! We are pleased to present you with the 2012 Microsoft® MVP Award!
This award is given to exceptional technical community leaders who actively share their high quality, real world expertise with others.
We appreciate your outstanding contributions in Visual Studio ALM technical communities during the past year.
This is incredibles news and I really want to thank both the people at Microsoft who nominated me and some of the (now) fellow MVP’s that I have worked with over the last year, both as part of the Visual Studio ALM Rangers program and as part of the TFS Build Extensions community project, in particular Mike Fourie and of course my colleague and main source of inspiration Terje Sandström 
I’m really looking forward to this year, it’s going to be a blast!
Friday, March 30, 2012
We have been using TFS 2010 build for distributing a build in parallel on several agents, but where the actual compilation is done by a bunch of external tools and compilers, e.g. no MSBuild involved. We are using the ParallelTemplate.xaml template that Jim Lamb blogged about previously, which distributes each configuration to a different agent. We developed custom activities for running these external compilers and collecting the information and errors by reading standard out/error and pushing it back to the build log.
But since we aren’t using MSBuild we don’t the get nice configuration summary section on the build summary page that we are used to. We would like to show the result of each configuration with any errors/warnings as usual, together with a link to the log file.
TFS 2010 API to the rescue! What we need to do is adding information to the InformationNode structure that is associated with every TFS build. The log that you normally see in the Log view is built up as a tree structure of IBuildInformationNode objects. This structure can we accessed by using the InformationNodeConverters class. This class also contain some helper methods for creating BuildProjectNode, which contain the information about each project that was build, for example which configuration, number of errors and warnings and link to the log file.
Here is a code snippet that first creates a “fake” build from scratch and the add two BuildProjectNodes, one for Debug|x86 and one for Release|x86 with some release information:
TfsTeamProjectCollection collection = TfsTeamProjectCollectionFactory.GetTeamProjectCollection(new Uri("http://lt-jakob2010:8080/tfs"));
IBuildServer buildServer = collection.GetService<IBuildServer>();
var buildDef = buildServer.GetBuildDefinition("TeamProject", "BuildDefinition");
//Create fake build with random build number
var detail = buildDef.CreateManualBuild(new Random().Next().ToString());
// Create Debug|x86 project summary
IBuildProjectNode buildProjectNode = detail.Information.AddBuildProjectNode(DateTime.Now, "Debug", "MySolution.sln", "x86", "$/project/MySolution.sln", DateTime.Now, "Default");
buildProjectNode.CompilationErrors = 1;
buildProjectNode.CompilationWarnings = 1;
buildProjectNode.Node.Children.AddBuildError("Compilation", "File1.cs", 12, 5, "", "Syntax error", DateTime.Now);
buildProjectNode.Node.Children.AddBuildWarning("File2.cs", 3, 1, "", "Some warning", DateTime.Now, "Compilation");
buildProjectNode.Node.Children.AddExternalLink("Log File", new Uri(@"\\server\share\logfiledebug.txt"));
buildProjectNode.Save();
// Create Releaes|x86 project summary
buildProjectNode = detail.Information.AddBuildProjectNode(DateTime.Now, "Release", "MySolution.sln", "x86", "$/project/MySolution.sln", DateTime.Now, "Default");
buildProjectNode.CompilationErrors = 0;
buildProjectNode.CompilationWarnings = 0;
buildProjectNode.Node.Children.AddExternalLink("Log File", new Uri(@"\\server\share\logfilerelease.txt"));
buildProjectNode.Save();
detail.Information.Save();
detail.FinalizeStatus(BuildStatus.Failed);
When running this code, it will a create a build that looks like this:
As you can see, it created two configurations with error and warning information and a link to a log file. Just like a regular MSBuild would have done.
This is very useful when using TFS 2010 Build in heterogeneous environments. It would also be possible to do this when running compilations completely outside TFS build, but then push the results of the into TFS for easy access. You can push all information, including the compilation summary, drop location, test results etc using the API.
Tuesday, February 21, 2012
A year ago I blogged about how to manage your build process templates using the TFS API. The main reason for doing this is that you can (and should!) store your “golden” build process templates in a common location in your TFS project collection, and then add them to each team project that requires those templates. This way, you can fix a bug or add a new feature in one place and have the change affect all build definitions.
However, by having the build process templates in a single location, the users must know where the build process templates are located and browse to that path and add it to the team project, before it will show up in the list of build process templates:
Unfortunately, you can’t manage the build process templates this way using Team Explorer, you have to resort to the TFS API to do these things.
Until now! 
In the latest release of the Community TFS Build Manager I have added support for managing build process templates.
The templates are accessible by selecting “Build Process Templates” in the “Show” dropdown:
This will show all registered build process templates, either in the selected team project or in all team projects, depending on your current filter:
All build process templates in the XDemo team project. The grid is of course sortable as the rest of the application. This lets you easily see where the template is registered.
Note that several of the build process templates in the list above is stored in the Inmeta team project, which is our team project for storing all artifacts related to our software factory, including the build process templates and custom activities.
Now, we can right click on a build process template and perform any of the following actions:
- Set As Default
This will set the selected build process template as the default build process template in the corresponding team project. There can only be one default build process template per team project, so the tool will automatically scan for any other default build process templates and set them back to “Custom”.
- Add to Team Project(s)
This will let you select one or more team projects where you want to add this build process template to:
In the list you can select one or more team projects. You can also specify that the template(s) should be set as default by using the checkbox “Set as Default”.
- Remove from Team Project(s)
This does the opposite from the previous operation, it removes the selected build process template(s) from one or more team project. After this operation, the template will not be visible in the “Build Process file” dropdown when editing a build process template.
Note: When removing a build process template, there might be build definitions using this template. If this is the case, the build manager will prompt you with a dialog before you proceed with the remove:
Hope that you find the new functionality useful. Please report bugs and feature requests to the Community TFS Build Extensions CodePlex site
Wednesday, February 01, 2012
The InvokeProcess activity is very useful when it comes to running shell commands and external command line tools during a build process. When it comes to integrating with TFS source control during a build, the TF.exe command line tool can be your friend, as it lets you do most of the usual stuff such as check-in, checkout, add, modify workspaces etc.
However, it can be a bit tricky to handle the output from tf.exe, since it often produces warnings that is not necessarily a problem for your build. This is not a problem related only to tf.exe, but to all applications that produces errors and warnings on the canonical error format.
The normal way to use the InvokeProcess activity is to setup the necessary parameters to call the tool with the correct path, working directory and command line switches. Then you add a WriteBuildMessage activity to the Handle Standard Output action handler and a WriteBuildError to the Handle Error Output action handler. In addition, you store the Result output property from the InvokeProcess activity in a workflow variable that you can evaluate after the InvokeProcess activity has finished.
This will output all standard output from the application to the build log, and all errors will be written as errors to the build log and will partially fail the build.
If you try this with TF.exe you will probably notice a problem with warnings from the tool being reported as errors in the build, causing it to partially fail the build, even though the ReturnCode was zero.
To solve this problem you need to collect the information that is passed to the Error Output action handler. Note that this handler is called several times so you need to handle formatting of the output in some way. Then, you check the ReturnCode from the InvokeProcess activity and in case this is <> 0, you write the collection information as an error to the build log (using WriteBuildError) and then throw an exception. Otherwise, just write the information to the build log using WriteBuildMessage, so you get all information out there.
The finished sample looks like this:
In the "Check out files” sequence I have defined a workflow variable called ErrorOutputFromTF of type string. In the “Handle Error Output” handler, I append the error to this variable, using the Assign activity:
I just append a newline character at the end to have all the errors on separate rows in the build log later. After the InvokeProcess activity I check the TFExitCode variable, that was assigned the ResultCode value from the InvokeProcess activity previously, if it is <> 0 I write the ErrorOutputFromTF to the build error log and then I throw an exception.
Here is a sample build log output:
Note that tf.exe in this case outputs information about check-in policies that have been overridden. This is an example of information that would cause the build to partially fail, but is now logged as information.
Sunday, January 22, 2012
I’ll be running a Team Foundation Server 2010 Deep Dive class twice this spring in Stockholm at our friends at Cornerstone.
The class is 4 days and we will be “diving deep” into all aspects of TFS 2010, including:
- Deployment
- Administration
- Source Control
- Check-in Policies
- Branching strategies
- Work Items
- Office integration
- Reporting
- Customization
- Team Build
- Working with build definitions
- Developing Custom Activities
- Automatic deployment
- Test Management
- Microsoft Test Manager
- Creating and running manual tests
- Automating manual tests/Coded UI Tests
- TFS/VS Extensibility
The class is a mixture of presentations and Hands On Labs, where you will get the chance to really get your fingers dirty with Team Foundation Server 2010.
See the full agenda and sign up here:
http://www.cornerstone.se/Web/Templates/CoursePage.aspx?id=2528&course=COUR2010083117182403594425&epslanguage=SV
If you are interested in Visual Studio ALM training but can’t make it to these sessions, please contact me and we can set something up that fits you and your company.
Friday, January 20, 2012
At my company we write a lot of tools and extensions that uses the TFS API to automate various things for us. A very common thing to automate is the creation of work items and the areas and iterations structure.
Creating a work item using the TFS API is simple, just connect to TFS, get the WorkItemStore service object and create a new work item and set any fields that you want to:
Creating a work item
//Connect to TFS and get the WorkItemStore object
var tfs = new TfsTeamProjectCollection(new Uri("http://localhost:8080/tfs"));
var wis = tfs.GetService(typeof(WorkItemStore)) as WorkItemStore;
//Get team project
var teamProject = wis.Projects["Demo"];
//Get the Bug Work Item Type
var wit = teamProject.WorkItemTypes["Bug"];
//Create a new Bug work item and set the title field
WorkItem wi = new WorkItem(wit);
wi.Title = "New Bug In New Area";
wi.Save();
Creating an area or iteration is equally simple:
Creating an area
//Connect to TFS and get the ICommonStructureService object
var tfs = new TfsTeamProjectCollection(new Uri("http://localhost:8080/tfs"));
var css = tfs.GetService(typeof(ICommonStructureService)) as ICommonStructureService;
//Get the root path of the new area
string rootNodePath = "\\Demo\\Area";
var pathRoot = css.GetNodeFromPath(rootNodePath);
//Create the new area, in this case it will be a new root area
css.CreateNode("NewRootArea", pathRoot.Uri);
BUT (yes there is a but, you could sense it coming), when you combine these two fellows into one task, e.g. create a new area (or iteration) and then create a new work item in that area, chances are high that you will receive the following exception:
Microsoft.TeamFoundation.WorkItemTracking.Client.ValidationException: TF237124: Work Item is not ready to save
at Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItem.Save(SaveFlags saveFlags)
at Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItem.Save()
The meaning of the error is not obvious, but if you would call the Validate() method before calling Save() (which you should, of course) you would see that it returns the Area field indicating that this field is the problem.
The underlying problem here is that in the TFS Data Store, work items and areas/iterations are persisted in two different stores. And these stores need to be synchronized before you can reference any new items that you added. You’ll notice the same issue in Visual Studio as well, when you create a new area or iteration in Team Explorer, you need to refresh Team Explorer in order to use the new nodes for work items.
But how do we do this programmatically? There actually two things that needs to be done:
- Request that the work item store is synchronized with the Common Structure store
- Refresh the local cache
Which is translated into the following code
Synchronize External Stores
//Synchronize the work item store with external stores (e.g. CSS)
private static void SyncExternalStructures(TfsTeamProjectCollection tfs, WorkItemStore wis,
ICommonStructureService css, string teamProject)
{
//Get work item server proxy object
WorkItemServer witProxy = (WorkItemServer)tfs.GetService(typeof(WorkItemServer));
//Get the team project
ProjectInfo projInfo = css.GetProjectFromName(teamProject);
//Sync External Store
witProxy.SyncExternalStructures(WorkItemServer.NewRequestId(), projInfo.Uri);
//Refresh local cache
wis.RefreshCache();
}
Call this method after creating the area/iterations and before creating the new work item and it will work as expected
Friday, December 30, 2011
The latest release of the Community TFS Build Extensions include a brand new tool called Community TFS Build Manager and has been created for two reasons:
- An implementation of the Team Foundation Build API which is referenced by the Rangers Build Customization Guidance V2 (available H1 2012)
- Provide a solution to a real problem. The Community TFS Build Manager is intended to ease the management of builds in medium to large Team Foundation Server environments, though it does provide a few features which all users may find useful.
The first version of the tool has been implemented by myself and Mike Fourie. You can download the extension from the Visual Studio Gallery here:
http://visualstudiogallery.msdn.microsoft.com/16bafc63-0f20-4cc3-8b67-4e25d150102c
Note 1: The full source is available at the Community TFS Build Extension site
Note 2: The tool is still considered alpha, so you should be a bit careful when running commands that modify or delete information in live environments, e.g. try it out first in a non-critical environment.
Note 3: The tool is also available as a stand alone WPF application. To use it, you need to download the source from the CodePlex site and build it.
Getting Started
You can either install the extension from the above link, or just open the Visual Studio Extension Manager and go to the Online gallery and search for TFS Build:
After installing the build manager, you can start it either from the Tools menu or from the Team Explorer by right-clicking on the Builds node on any team project:
This will bring up a new tool window that will by default show all build definition in the currently selected team project.
View and sort Builds and Build Definitions across multiple Build Controllers and Team Projects
This has always been a major limitation when working with builds in Team Explorer, they are always scoped to a team project. It is particularly annoying when viewing queued builds and you
have no idea what other builds are running on the same controller. In the TFS Build Manager, you can filter on one/all build controllers and one/all team projects:
The same filters apply when you switch between Build Definitions and Builds. In the following screen shot, you can see that three builds from three different team projects are running on the same controller:
You can easily filter on specific team projects and/or build controllers. Note that all columns are sortable, just click on the header column to sort it ascending or descending.
This makes it easy to for example locate all build definitions that use a particular build process template, or group builds by team project etc.
Bulk operations on Build Definitions
The main functionality that this tool brings in addition to what Team Explorer already offers, is the ability to perform bulk operations on multiple build definitions/builds. Often you need to modify or
delete several build definitions in TFS and there is no way to do this in Team Explorer.
In the TFS Build Manager, just select one or more builds or build definitions in the grid and right-click. The following context menu will be shown for build definitions:
Change Build Process Templates
This command lets you change the build process template for one or more build definitions. It will show a dialog with all existing build process templates in the corresponding team projects:
Queue
This will queue a “default” build for the the selected build definitions. This means that they will be queued with the default parameters.
Enable/Disable
Enables or disables the selected build definitions. Note that disabled build definitions are by default now shown. To view disabled build definitions, check the Include Disabled Builds checkbox:
Delete
This lets you delete one ore more build definitions in a single click. In Team Explorer this is not possible, you must first delete all builds and then delete the build definition. Annoying! 
TFS Build Manager will prompt you with the same delete options as in Team Explorer, so no functionality is lost:
Set Retention Policies
Allows you to set retentions policies to several build definitions in one go. Note that only retention policies for Triggered and Manual build definitions can be updated, not private builds.
This feature also gives you the same options as in Team Explorer:
Clone to Branch
My favorite feature Often the reason for cloning a build definition is that you have created a new source code branch and now you want to setup a matching set of builds for the new branch. When using the Clone build definition feature of the TFS Power Tools, you must update several of the parameters of the build definition after, including:
- Name
- Workspace mappings
- Source control path to Items to builds (solutions and/or projects)
- Source control path to test settings file
- Drop location
- Source control path to TFSBuild.proj for UpgradeTemplate builds
All this is done automagically when using the Clone to Branch feature! When you select this command, the build manager will look at the Items to build path (e.g. solution/projects) and find all child branches to this path and display them in a dialog:
When select one of the target branches, the new name will default to the source build definition with the target branch name appended. Of course you can modify the name in the dialog. After pressing OK, a new build definition will be created and all the parameters listed above will be modified accordingly to the new branch.
Bulk operations on Builds
You can also perform several actions on builds, and more will be added shortly. In the first release, the following features are available:
Delete
This will delete all artifacts of the build (details, drops, test results etc..). It should show the same dialog as the Delete Build Definition command, but currently it will delete everything.
Open Drop Folders
Allows you to open the drop folder for one or more builds
Retain Indefinitely
Set one or more builds to be retained indefinitely.
Bonus Feature – Generate DGML for your build environment
This feature was outside spec, but since I was playing around with generating DGML it was easy to implement this feature and it is actually rather useful. It quickly gives you an overview of your build resources, e.g. which build controllers and build agents that exist for the current project collection, and on what hosts they are running. The command is available in the small toolbar at the top, next to the refresh button:
Here is an example from our lab environment:
The dark green boxes are the host machine names and the controller and agents are contained within them.
Note: Currently the only way to view DGML files are with Visual Studio 2010 Premium and Ultimate.
I hope that many of you will find this tool useful, please report issues/feature requests to the Community TFS Build Extensions CodePlex site!
Friday, December 16, 2011
Brian Harry just posted an update on the latest version of the TFS 2010 Power Tools. This will most likely by the last version of the Power Tools for the TFS 2010 version, next version will target Dev11!
The main improvements in this release are:
- Team Foundation Server Power Tools for Eclipse
- MSSCCI Provider for 64-bit IDE’s
- VS 2010 Power Tools update
- Improved Work item Search
- Best Practice Analyzer now also analyzes the integration with Project Server, if you are using it
Check out the early Christmas gift here:
http://blogs.msdn.com/b/bharry/archive/2011/12/16/december-2011-tfs-power-tools-release.aspx
Monday, December 12, 2011
I often use the VS 2010/TFS 2010 evaluation virtual machines that Microsoft publishes every 6 months with the latest bits. It’s a great timesaver to use an image where everything is already setup and also contains a bit of sample data that is useful when you want to demo something for customers.
There is one thing that has always been a, albeit small, but still very annoying problem and that is that the builds always partially fail when you start using the image. When you want to demo the powerful feature of associated work items in a build, you’ll find yourself with your pants down since the build fails when trying to update the associated work item! Even when looking at the historical builds for the Tailspin Toys project, you will notice that they also partially failed:
If you look at the error message in the build details, you’ll see the following error:
The work item 'XX' could not be updated: 'TF237165: Team Foundation could not update the work item because of a validation error on the server. This may happen because the work item type has been modified or destroyed, or you do not have permission to update the work item.'
The problem here is that the build agent by default is running as the NT AUTHORITY\SYSTEM account, which is an account that do not have permission to modify work items. Your best option here is to switch account and use the Network Service account instead. Open TFS Administration Console, and select the Build Configuration node. Press the Stop link in the Build Service section:
Select Properties and select NT AUTHORITY\NetworkService as Credentials
Press Start to start the build service with the new credentials.
If you would queue a new build now, the build would fail because of conflicting workspace mappings. The reason for this is that we haven’t changed the working folder path for the build agents, so when the build agent try to create a new workspace, the local path will conflict with the workspace previously created by NT AUTHORITY\SYSTEM.
So to resolve this we can do two things:
- (Preferred). Delete the team build workspaces previously created by the SYSTEM account. To do this, start a Visual Studio command prompt and type:
tf.exe workspace /delete <workspacename>;NT AUTHORITY\SYSTEM
If you need to list the workspaces to get the names, you can type:
tf workspaces /owner:NT AUTHORITY\SYSTEM /computer: <buildserver>
- (Less preferred, but good if you want to switch back later to the old build service account)
You can also modify the working folder path for the build agents, so that they don’t conflict with the existing workspaces. Click Properties on the build agent(s) and modify the Working Directory proprerty:
b
In this case, you can for example change it to $(SystemDrive)\Builds\NS\$(BuildAgentId)$(BuildDefinitionPath) where NS = Network Service.