Geeks With Blogs


Dylan Smith ALM / Architecture / TFS

I've had a little frustration lately with Team System as a result of not using the complete Team Suite version.  When the licensing for Visual Studio changed last year to the role-based versions and/or Team Suite, we considered the options and decided that VSTS For Developers should be all we need.  We do not do any web or load testing (just unit tests), and we do not have a need for the designers available in the Architect version.

Since then that has held true for the most part.  Except for some important scenarios.  Most of my problems arose when I tried to get a Team Build going.  I managed to get the build itself running.  But then I wanted to enhance it to run our tests as BVT's (Build Verification Tests).  Based on what I'd read, I had believed this was possible.  After all, creating unit tests is part of the capability of VSTS For Developers (you just have a limited amount of control over how you organize them compared to Team Test).  Well it turns out that it's not so fact it doesn't appear to even be possible.  I installed VSTS for Developers on my build machine, and updated my Team Build to run some tests. But I soon discovered that in order to have the Team Build run some tests you need to specify the test list to use (all I want to do is have it run ALL my tests...why should I have to create a test list for this?).  Well creating test lists is a feature limited to Team Test and not available in VSTS For Developers.  This was confirmed by Eric Jarvi in this forum post:

It turns out that their is no official support for creating a Team Build that includes BVT's if all you own is VSTS For Developers. I believe it may be possible by manually editing the .vsmdi file which is just XML and creating a test list in raw XML.  Rather than mess with that, I decided to just fire up a VPC, install Team Suite Trial, grab a copy of my project from source control, add a test list, then check it all back in.  Then I should just be able to use that test list for my Team Build right?  Sure I'd have to go into my VPC and add in all my new tests every once in a while, and what would I do once the Trial expired, but at least I could get something running right now.

Well sadly not even that would work.  Once I checked it in from the Team Suite edition and tried to run my team build, now it is complaining about all kinds of test related dependencies that aren't available on the build server (because it only has VSTS For Developers).  I played with this for a while, trying to manually edit some of the XML files to remove the dependencies, but after an hour or two of trying I finally got frustrated and backed out my changes in source control.

I still don't have a Team Build running that will run my unit tests.  Before doing anything drastic like abandoning MSTest and Team Build, I have another strategy I want to try.  I think I might be able to modify the MSBuild script used so that it invokes MSTest directly.  Failing that, I'm pretty certain I can use CruiseControl and MSTest.exe as a CruiseControl task.  But in that case I think I might lose the code coverage data.

Anyways, I had somewhere I was meaning to go with this rant.  Most of these difficulties could have been avoided if I was using Team Suite everywhere.  I'm sure many others have experienced similar frustrations or confusion as to what they can and cannot do with the role-based versions of Visual Studio.  My suggestion to Microsoft, is why not do away with the role based versions.  Drop the price of Team Suite to somewhere between what the cost of a role-based version is and Team Suite is today (closer to the role cost than the suite cost).  What you lose in margin you should make up for in volume.  Sure the price to customers will go up a bit, but I think the added cost will be worth eliminating the frustration from trying to get the role versions to do something they weren't meant to do, and the confusion over what it is each one is meant to do and is not meant to do.

I can foresee this problem getting worse once VSTS For DB Pro's comes out.  What if I buy VSTS For DB Pro for my DBA, and VSTS For Developers for my devs.  I have unit tests that my developers write that hit against the database, these unit tests need to deploy the db schema as part of the test.  Will they be able to compile and deploy the database from the dbproj within VSTS For Developers?  It doesn't appear so (at least not until a new version of VSTS For Developers is released).  I understand not allowing them to maintain the schema, you should need VSTS For DB Pro's for that, but the developers will need to be able to deploy that schema in order to run their unit tests.  And to deploy the schema, they should probably also be able to build the dbproj to generate the latest version of the schema.  Even if you ignore the database deployment with unit test issue.  My understanding of MS's vision (and I share in this vision) is that every dev should be running a local instance of the database, and the master version is kept in source control and maintained by the DBA.  Well when the DBA makes updates to the schema how is the developer supposed to deploy those changes to their local database?  My understanding is that VSTS For Developers won't be able to recognize a dbproj at all, let alone compile and deploy it.  Will it be the DBA's responsibility to deploy the database to every developers SQL instance?  How will he know when it is safe to do so?  Will the dev's have to explicitly request the DBA to push out the changes on a workstation by workstation basis when they are ready for a new version?  Not only that (and I may be completely wrong here), but isn't SQL Server Developer Edition restricted to only support local connections.  If so the DBA wouldn't be able to remotely deploy the database to the dev workstations even if he wanted to.

Don't misunderstand me, I am a HUGE fan of the new team system functionality.  If I wasn't I wouldn't get so worked up over these issues.  I'm just starting to feel some frustration with limitations that I don't agree with, and I only see it getting worse down the road if something isn't done to address/improve the issue.  If anybody from the VSTS team is reading, please consider this feedback and think about ways to avoid these problems.  As I said above, I'm all for eliminating the role-based versions and adjusting the pricing appropriately so everybody is either on VS Pro, or VSTS Team Suite.  If that's not an option, then at a minimum consider what common scenario's teams will be trying support when they are using the role-based versions and make sure that enough functionality is included in each version to support those scenarios.  Especially when VSTS For DB Pro's becomes available, I foresee some major hurdles for teams that are using VSTS For Developers for the devs and VSTS For DB Pro's for the DBA's.  To solve the Team Build issues, another possible solution is to provide a special VS install for build servers, that doesn't support development activities, but can support Team Build's that include the features from all the different role versions (build dbproj's, run tests, etc).

Posted on Thursday, August 17, 2006 12:09 PM | Back to top

Comments on this post: Team System "Role" Versions - Frustrations / Suggestions

# re: Team System "Role" Versions - Frustrations / Suggestions
Requesting Gravatar...
Hi Dylan,

Unfortunately, as you have confirmed by Eric Jarvi, creating test lists was the key problem. The good news is that (assuming you own dev edition), you already have the necessary software to get your build runnning one you have the test list.

You are allowed (and basically required) to install a VS Test or Dev edition (both of which support UT and Code Coverage) on the build server. The Team Edition license you have allows dual use with interactive on one system and build-only on another. Skim the install docs for team build for more details.

There used to be some blog posts on how to just copy the required DLLs back in the beta days of VSTS, so if you're adventurous and don't want to install VS on your build server, you might look there. Cheers!
Left by Dave McKinstry on Aug 21, 2006 6:45 AM

# re: Team System "Role" Versions - Frustrations / Suggestions
Requesting Gravatar...
Even more frustrating about the team role editions, the four types of SOA diagrams that you create in Team Architect cannot even be viewed by other team members unless everyone is using Team Architect or Team Suite. To combat the problem, the Team Architect user must export the diagrams as images and include those as project files so that everyone can understand the architecture. These images must continue to be updated! THanks for the great article.
Left by Davin Mickelson on Sep 05, 2006 8:52 AM

# re: Team System "Role" Versions - Frustrations / Suggestions
Requesting Gravatar...
It looks you are also licensed to install the dev edition on the build server as long it runs non-interactively. So you may not have to do so many workarounds. Admittedly it would be nice to have it there as part of the build server.
Left by barry on Oct 17, 2006 9:47 AM

# re: Team System "Role" Versions - Frustrations / Suggestions
Requesting Gravatar...
We are encountering the same issues at my company. We have many developers that straddle multiple roles. For example, we have software developers who almost always build stored procedures. We tried to use the DBA schema management features of VSTS and were expecting to be in our glory.

Come to find out, the developer version of VSTS can't even view the source controlled schema. Only DBA and Architects can see it at all! This is hugely frustrating. I want to be able to have any developer check out a stored procedure and make a change to it and check it back in. No can do?

I'd go for the advance in pricing to have people be able to work in multiple roles, too. Maybe MS can allow floating role licencing, so you can use only one role at a time, but allow developers to switch roles via some UI control. The real world often has developers, and architects; testers and DBAs sharing and filling multiple roles. They have to find a way to make VSTS work for companies other than those with absolute invariable division of labor or unlimited software budgets.

Yikes! I hope Microsoft is listening ...
Left by trw on Aug 26, 2008 3:39 PM

# re: Team System "Role" Versions - Frustrations / Suggestions
Requesting Gravatar...
Yeah, it seems that in order to take advantage of alot of Team System features you really do need Team Suite. In the conversations I've had with Microsoft folks they don't seem to be placing a high priority on resolving this issue in the near future.

My suggestion to everybody is to realize that if you want to take advantage of the Team System features you almost have to pony up the extra cash for the full Team Suite.
Left by Dylan Smith on Aug 26, 2008 4:02 PM

Your comment:
 (will show your gravatar)

Copyright © Dylan Smith | Powered by: