Dylan Smith

Traditional estimation involves breaking down the work into tasks, then assigning an estimate of effort in hours/days/etc to each task.  This is then combined into a gantt chart or something similar to create a schedule of the work to be done.

Agile estimation doesn't do estimates at the task level, but rather focuses on the throughput of the team over a fixed period of time.  If the team uses an iterative approach to development the iteration length is a natural time-box to use to measure throughput/velocity, but even if a non-iterative approach is used an arbitrary time-box works just as well (eg. 1 month).  The work to be done also needs to be able to be broken down into "units" of work.  You will want to use something that is easy to break it down into with a minimum of effort/analysis.  Some common units of work are features, bugs, use cases, user stories, etc.  Something such as classes, lines of code, # of methods are typically poor choices as they require a non-trivial amount of effort/analysis to come up with up-front, and even then are typically wildly inaccurate.

Once you've decided on the time period to use, and the unit of work metric to use the only estimating you need to do is to estimate the "velocity" of your team in units of work / time period.  For example, if the team is responsible for fixing defects you would estimate in terms of how many defects per month you can resolve.  Once you have been doing this for a number of months you can simply use the historical velocities from previous months to figure out the average velocity and std deviation to allow you to make an estimate along the lines of "there is a 90% chance we can resolve 25+ defects in the next month".  Or you can make an estimate something like "there are currently 100 defects in the backlog, at our estimated velocity there is a 90% chance we can finish all 100 in 4 months time".  Another common scenario is if you have a target deadline, say a major release in 2 months, you are able to make a statement to the effect "in the 2 months left before the release we estimate - with 90% confidence - that we can resolve 50 of the 100 defects in the backlog, we'll need the stakeholders to prioritize the defects and identify the 50 they wish to make it into the next release".

A common reaction is to think that some defects/features take more time than others, how can such an broad estimation possibly be accurate enough?  But in practice if you are calculating velocity on a batch of work units (eg. The amount of defects resolved by a 3 person team over a month) then the variability tends to average out (the law of large numbers), with the especially long work units being offset by the shorter work units, and if you were to measure estimated velocity (from historical data) against actual velocity you'd find that it is much more accurate than if you were to measure estimated time per task against actual time per task (I believe that studies like this have been done, if anybody has a link to something like that please leave a comment).

There are lots of other benefits to this approach, one is that it provides a concrete source of feedback/input to assess schedule impact - being the velocity measurement at the end of every time period.  For instance, if you have 100 defects in the backlog and have planned on resolving them all over 4 months (based on historical/estimated velocity), at the end of the first month you will have a new velocity metric that you can use to adjust your plan if necessary.  If the measured velocity at the end of the first month is only 10 defects/month then as a PM you will have to take some action to either adjust the plan out to 10 months instead of 4, or do some investigation as to why the velocity has dropped off so drastically and remove some of the impediments to the team to allow them to regain their historical 25 defects/month velocity (and then adjust the schedule as necessary).  This turns out to be a much more reliable process for updating the schedule/plan than relying on developers to provide feedback when they realize an estimate won't be met.  Developers tend to be overly optimistic, and even if they realize that they are 4 days into a 6 day task and not even halfway done yet, rather than having to tell the PM to adjust the estimate, they tend to try to work harder/longer/smarter to squeeze the rest of the work into the remaining 2 days.  This typically results in them waiting until they are almost at the 6 day mark before finally "admitting defeat" and informing the project manager that the estimate won't be met.  More importantly is the fact that the schedule estimates can only really be updated by the developers once they actually start work on the tasks, so after the first month of work you only have estimate updates on 25% (or less) of the tasks, and usually the other 75% of the estimates are left in tact since they haven't been started yet (a strong PM may notice a trend and extrapolate that trend out to adjust the estimates on the other 75% of the work, but in my experience that doesn't happen too often).  Whereas with the velocity approach, it's very intuitive that if you notice your velocity is only 10 vs the 25 estimated this month it makes sense to expect that it will remain at 10 for the upcoming months (unless something is done to affect it, eg. Add more resources, remove distractions, etc).  Then the replanning effort is a simple division of the # of work units divided by the expected velocity, and voila you have your expected project length.

In my experience that is really the information that is needed to perform planning activities, and estimating at the individual task level is usually inaccurate, time-consuming, frustrating, and usually totally unnecessary.

I've been thinking about what makes one chunk of code "better" than another. And in general what qualities do I look for and focus on when writing code.

Personally I believe that the 2 most important qualities of good code (in order of priority) are:

1. Meets customer requirements
2. Maintainability

Now those are pretty broad qualities, but I think I can break it down a little further. How do I focus on meeting customer requirements? Well I believe there are 2 major facets to this, in order of priority:

1. Verifiable Requirements - This usually means executable requirements in the form of acceptance tests. This provides us with a verifiable way to determine if our implementation meets the requirements as stated, whether they be functional or non-functional requirements. However, it doesn't address the issue of inaccurate requirements, thats the purpose of the next area of focus.
2. Short Feedback Cycle - When implementing an application there should be a strong focus on shortening the feedback cycle. This means frequent releases, and high-quality releases. This enables you to identify inaccurate requirements and complements the Verifiable Requirements focus. A large part of accomplishing this lies in maintaining a high-level of quality such that it is always as close as possible to release-ready. This typically involves techniques such as continuous integration, aggressive refactoring, and comprehensive unit tests.

How do I define maintainability? I see it as the combination of 2 sub-qualities, in order of priority:

1. Testability - I believe that testability is the biggest factor in determining whether code is maintainable. Whether you have an existing test suite or not is a separate issue, but when evaluating the implementation code itself testability is key. This means using principles such as separation of concerns, dependency injection, the law of Demeter, loose coupling, etc.
2. Simplicity - After testability the next most important factor in determining maintainability is simplicity. Assuming you have multiple choices for how to implement a feature, and they are all equally testable, the simplest solution wins.

Obviously just stating that the simplest solution is best is a little vague. So lets break down what I mean when I say "simple". I think that simplicity is determined by 3 factors, in order of priority:

1. Separation of Concerns - Each piece of code should have one and only one well-defined responsibility. This allows you to easily isolate pieces of logic for testing. It also helps lead to a set of classes that has high-cohesion and loose-coupling, an important set of design principles.
2. Minimize Duplication - Also commonly called DRY (Don't Repeat Yourself), this demands that you employ aggressive refactoring to drive out duplication in your implementation. Duplication occurs in many forms, and spotting all forms of duplication is a skill acquired with experience. Nonetheless, minimizing duplication will result in a simpler solution that is ultimately more maintainable.
3. Minimize # of classes/methods - Assuming you've first focused on separating the concerns and minimizing duplication, the last factor you should consider is minimizing the # of classes/methods while still retaining the previous 2 qualities. This also falls under the YAGNI (You Ain't Gonna Need It) principle, since if classes/methods don't help to separate concerns or reduce duplication, and the current customer requirements can be met with less classes/methods, then you should use the solution with the fewest moving parts.

So to summarize, I believe that the hierarchy of qualities that are important for code (in order of priority) is:

• Meets Customer Requirements
• Verifiable Requirements
• Short Feedback Cycle
• Maintainability
• Testability
• Simplicity
• Separation of Concerns
• Minimize Duplication
• Minimize # of classes/methods

How would you define "good" code?

I've recently been involved with a project that involves a BizTalk 2006 application.  I've been focusing most of my time on improving the testing infrastructure, and trying to automate alot of the deployment and configuration tasks with the goal of getting to the point where we can setup a Continuous Integration environment.

I'm happy to say that I think we're pretty much there, but not without having to overcome some hurdles along the way.  This is my first encounter with BizTalk and it was a great learning experience.  What I realized early on is that BizTalk doesn't appear to be very tester-friendly.  I'm a TDD guy and I can see that trying to test-drive a BizTalk orchestration could prove tricky. It just seems like it behaves like too much of a black-box, and trying to develop in small increments and isolate specific functionality in the BizTalk components doesn't appear to be easily possible.  I find that testability comes from being able to have control over the component interactions, and the ability to monitor what is taking place.  When dealing with code projects this is typically achieved through loose-coupling, dependency injection, and/or mock objects.  With BizTalk your options are somewhat more limited.

I did manage to stumble onto a project called BizUnit which tries to tackle that problem.  I was brought into this project after the bulk of the BizTalk functionality was already implemented (although without any tests) so I haven't had the opportunity to try it out yet.  If anybody out there has experiences with it I'd love to hear what you think.  Or just how you tackle unit testing BizTalk in general.

Aside from being able to unit-test and TDD the BizTalk implementation, the other challenge has been trying to get a good testing infrastructure setup that is automated, fast, and easy to run.  The idea is that once we have everything setup, we should be able to setup a CI environment that can compile and run the test suite automatically on check-in's.

The main issue has been around the deployment of the BizTalk components.  In order to run any tests against the BizTalk projects they first need to be deployed into BizTalk (and the old versions undeployed).  Deployment is (or was) a manual process involving stopping the old application in BizTalk Admin, deleting the old application, deploying the new version from VS, importing the bindings in BizTalk Admin, and starting the application in BizTalk admin.  And this all had to take place any time any changes were made to the BizTalk projects.

My first thought was that surely all of this could be done from the command-line, all I need to do is throw together a few commands into a batch file or msbuild file to automate this process.  Unfortunately it's not that simple.  About the only thing from that process that can be done from the command-line is importing the bindings file.

Luckily I found the Enterprise Solutions Build Framework from Microsoft Consulting Services that promised to make life easier for me.  It is a set of MSBuild tasks that includes tasks for stopping and starting BizTalk applications, deleting and creating BizTalk applications, etc.  Exactly what I was looking for.Turns out the GotDotNet site hasn't been updated in over 6 months, and most of the links - including the downloads link - are broken.  I managed to find another site that has a sample project available for download that includes the SBF binaries.  It also includes a great sample of using the BizTalk tasks to uninstall and redeploy a BizTalk solution.  I modified the sample for my own use, the only really significant change was replacing his bindings stuff with an exec task that called out to BTSTask to import the bindings (the SBF includes a task to import the bindings, but neither the author of that sample nor myself could figure out how to get it to work).

Here is the MSBuild script that we are using to deploy our BizTalk components (client-identifying information replaced with *'s).  I have a bit of code that runs this in the ClassInitialize of my test fixture.

I believe in having 3 layers of testing: Unit Tests, Acceptance Tests, and Exploratory Testing.  Each of these layers is somewhat independent of the other and each layer alone essentially attempts to validate that the entire system works.  However, none of these layers alone is perfect, but having all 3 in place simultaneously makes the situation much better.

Unit Tests
Unit tests are the tests that the programmer writes as he develops the software.  We develop using TDD and the unit tests are automatically generated as part of that process.  Unit Tests should focus on small pieces of functionality, isolating it from other related functionality.  Typically I accomplish this using mock objects to isolate classes or methods from it's dependencies by mocking the dependencies.  If TDD is performed effectively your Unit Tests should automatically provide a very high level of code coverage (90%+ as a rule of thumb).

Ex. I have an UpdateCustomer method I need to implement.  I will have a Unit Test for the "happy case", I will have a unit test passing in a blank customer name (invalid data), I will have a unit test passing in a duplicate customer name (invalid data), and unit tests for any other validation tests that I need to "test-drive" into the implementation.  I will mock out the DatabaseLayer, and any other dependencies the UpdateCustomer method may use to perform validation or other logic.

Acceptance Tests
Acceptance tests are higher level tests that are written from a customers perspective.  They are not concerned with isolating small pieces of functionality like unit tests but instead test the whole system from end-to-end.  If you ever develop "user stories" as part of your requirements (eg. XP) or a scripted test to be manually executed by a human, then an acceptance test is basically an automated version of those.

Eg. For implementing a customer maintenance application I may have an acceptance test that creates a new customer, updates a field on it, retrieves the customer and validates that the properties match what I expect, then does a delete and another retrieval to validate that the delete was successful.

Exploratory Testing
This is testing that is done manually by a human.  This is probably what most people are used to.  It does not use any test script, it is just simply a person using the system and trying to discover new bugs.  In my experience a human is usually very good at identifying weak portions of the system, and then reasoning where other weaknesses may exist based on what they've seen.  I use this in addition to Unit Testing and Acceptance Testing to help us find any bugs the automated tests may have missed.

Now that I've explained at a basic level what each of the test types I use are, lets take a look at how I use these 3 types of testing in concert.

Typically when I'm developing an application I'll start by designing what I think the UI should look like and how it should function (we currently do not write any automated tests directly against the UI layer).  This ensures I'm focusing first and foremost on the user experience and meeting the users needs; since the UI is the only portion of the application the user sees or really cares about.  I'll draw up the UI and write up the code that interacts with my business layer (typically a web service).  In the process of doing this I discover (aka design) how I want/expect the interface on the middle-tier to look like to make it easy for the UI to consume.

Obviously the UI won't compile yet, as the Web Methods I have been coding against and designing as I go don't exist yet in the middle-tier.  So the next step is to create skeleton methods in the middle-tier (Throw New NotImplementedException); just the bare minimum of code needed to get everything to compile.  At this point I'll usually check my code into source control since it is at a compilable state and no failing tests (since I haven't written any new tests yet - that's next).

My next step is to write the first Acceptance test.  This will exercise the web methods from the middle-tier that we consumed while coding the UI.  Once I get a reasonable Acceptance test written up, it should compile, and fail (since all the web methods should be throwing NotImplementedExceptions currently).  This Acceptance test represents the first "user story" that I'm going to focus on implementing.  Now that I have a goal (getting the Acceptance test to pass) I'll start implementing the middle-tier methods using TDD to write unit tests and implementations as I go.

I'll keep using TDD to drive out the implementation necessary to get the Acceptance test to pass.  Eventually I'll get to the point where I have enough implemented that my Acceptance test will pass.  At this point I should have the one passing Acceptance test and quite a few passing unit tests as a result of the TDD.  At this point I'll check in my code into source control (good time to do it since all tests are passing).

The next step is to write another Acceptance Test exercising some more functionality that the first Acceptance Test didn't cover.  Then I'll TDD the implementation required to get this Acceptance test to pass.  Then check-in to source control.  This process repeats until I'm satisfied that I have enough Acceptance tests to cover the functionality required. 

When I use TDD to drive the implementation I stick to the rule that I'm only implementing strictly what is required to make the test pass.  If I feel there's more logic needed in the implementation, I won't write it until I first have a unit test to validate it.  However, when I'm using TDD to drive the implementation of a specific Acceptance test, I do not stick strictly to what I need to make the Acceptance test pass.  For instance, lets say I have an Acceptance Test that calls UpdateCustomer() web method.  While I'm TDD'ing the implementation of this UpdateCustomer web method I'll create unit tests to drive out all the functionality I believe is required in that method.  For example, I may have a handful of unit tests that just call UpdateCustomer() passing in various different types of invalid data.  This will drive the implementation of the validation logic in UpdateCustomer.  I  write these unit tests and implementation, even if that logic isn't specifically exercised by the Acceptance Test I'm currently trying to implement.

Once I have all the Acceptance Tests I feel necessary written and passing then I should be done implementing everything for this portion of the UI as far as I'm concerned.  Now it's time for some exploratory testing.  I will typically perform at least a little bit of exploratory testing myself just to ensure that the UI works properly and everything I needed to implement is in fact completed.  Then you should have somebody with fresh eyes do some more exploratory testing (ideally a dedicated tester in conjunction with the end users).

Anytime a bug is found while performing exploratory testing this indicates that one or more automated tests are missing from the test suite.  There should be at least one unit test to demonstrate the bug, and depending on the nature of the bug you may wish to write an acceptance test in addition to the unit test to demonstrate the bug.

Another thing I would note, is that if anytime during the implementation or maintenance of the application you wind up with a failing acceptance test (that was previously passing) but no failing unit tests, the first step should be to write a failing unit test, make the unit test pass, then check to see if the acceptance test is now passing.  If not write another unit test and repeat.

I feel that having these three layers of testing provides the safety net that enables pain-free refactoring, along with a strong focus on quality code.  Unit Tests take care of testing the basic pieces of logic in isolation, Acceptance Tests ensure that all that logic integrates properly and does in fact work together as a complete system.  Exploratory Testing helps to capture all the bugs and logic that the developer forgot about while writing the unit and acceptane tests.

Let me know what type of testing strategy you follow?  Do you think this sounds like too much effort?  If so what would you do differently?

I was chatting with a buddy of mine who works in consulting about the Project Status Report I've been using (see previous entry).  He was concerned that something like that wouldn't work for a consultant (both the report and the process itself) without some modification.  His main concern was that they need to track billable hours for both project costing and billing the client.  My process and report don't deal with hours instead choosing to deal with "work points".  Work Points represent the amount of effort required relative to the other features on the list.  If you wanted to I'm sure you could spend some time and track hours worked and then calculate out how many hours worked correspond to each work point.  In my circumstances though that is not needed.  Work Points gets me an abstract velocity number which allows me to easily detect if we are speeding up or slowing down our progress, and allows me to calculate an estimated completion date which is really all I'm interested in.

For a consulting company, they are interested in hours for calculating costs and billing clients.  We were talking about how this report and process could be adapted to work in his situation.  I thought about it for a while and have an idea that is probably pretty out there (but hey, these are the things I like to think about).

Instead of charging clients by the hour why not charge by the work point.  It would probably be more attractive for the clients (once they got used to it) since now they are paying for value delivered rather than hours worked.  And the cost won't fluctuate depending on how many hours it takes (although I'm guessing consultants provide hours estimates up front that the clients are charged for regardless of the actual hours).  I can see an advantage to consultants also, but let me rehash a conversation I had with another consulting buddy a while ago.

I was talking with Joel Semeniuk about his consulting company (Imaginets) we were talking about a situation he encounters there. Imaginets has extremely low employee turnover.  I take this to be a sign that they know what they are doing and are a great company to work for.  However, the problem this presents is that their employees are getting more and more senior as they gain experience and demanding higher and higher salaries.  What is the company to do?  Do they start charging clients higher and higher rates?  That may be one option, and you could certainly argue that it's justified.  Joel has some novel ideas on how to deal with that situation, but lets get back to the original conversation.  If you were to charge clients based on work points instead of hours worked it seems like it would help in this situation.  The client and the consultants don't need to deal with how to charge for senior devs vs junior devs and all those inbetween.  If they just charge based on work points it's up to the consultants how to staff the project.  If they put junior devs on the project the assumption is that they would have a lower velocity than a senior dev but it would even out because their salary is presumably lower to compensate.  Likewise, paying the senior devs more would be justified assuming that their velocity is high enough to justify it.  All the consulting company would need to worry about is that the aggregate velocity is sufficient to hit the date given to the client.

It also removes the task of having to provide up front estimates in hours of how long everything will take.  Instead the estimates can be done in abstract work points which in my experience are infinitely easier to estimate.

Now my knowledge of consulting companies and their processes and issues is somewhat limited, so there's a good chance this is a crazy idea.  If so give me some feedback and let me know the many reasons why it won't work.  Or maybe it would work, maybe lots of consulting companies already do something similar and I'm just not aware of it (easily possible).

I wanted to take a few minutes to talk about the project management / tracking / estimation methods I use.  Over time the processes and tools I use have changed quite a bit, but the way I've been doing things recently really works well for me.  The primary tool I use to manage a project is a feature list.  Similar to a product backlog from SCRUM (although we don't do sprints...or I suppose you could say we do a sprint for every feature).  The feature list is constantly evolving as we discover new requirements, or new feature requests are received.  I maintain this list using a Sharepoint list on our project site.  The feature list is prioritized (stack-ranked) in order from highest priority to lowest.  We don't give each feature a priority from 1 to 10 or some similar ranking (low, medium, high, etc), instead we sort the whole list according to priority relative to the other items on the list.  This prioritization is done as a group with the project stakeholders, and we hold a weekly meeting where any new feature requests are reviewed and their priority relative to the rest of the list determined.  One of the nice things about using this method, is that we never have to refuse a feature request, or somebody requesting to expand the scope.  Every request will be added to the list, then it's up to the group to decide where it should be prioritized.  Even feature requests that are just wish-list type stuff, that we know won't get done in the near future can still be added to the list, they will just be prioritized near the bottom.

At our weekly meeting, in addition to reviewing and prioritizing new feature requests, we also review and update (if necessary) the "go live" point.  This is the cutoff point in the feature list that all the stakeholders feel includes enough functionality and benefits that we can roll out the application into production.

Each item in the feature list is also assigned an Effort Required rating from 1 (easy) to 5 (hard).  This is just a rough estimate done by me (the development manager) that gives everybody an idea of the work required to implement each feature relative to the other features.  If I find myself giving a rating of 5 (hard) I will usually attempt to break the feature up into 2 or more smaller features that can be tackled separately.  If I feel a feature is a 1 (easy) I will see if I can group it together with another related feature.

Along with each feature we also maintain a % Complete which is updated by the developer(s) working on that feature.  We define specific percentages to represent key points (milestones) in that features lifecycle.

0-49% is under development
50% is ready for review by the feature owner(s)
60% means it has been reviewed
61-79% means it is undergoing revisions as a result of the review
80% is ready for demo to the larger group (at our weekly meeting)
90% means it has been demo'd
91-99% is undergoing final revisions as a result of the demo
100% is completed

Now that we have the effort required rating for each feature (I also call these "Work Points") and the % complete, we can do a number of useful calculations.  We can multiply out the % complete by the Work Points for each feature and sum it up to get the total work points completed.  If we compare this to the total work points required we can come up with a % complete for the whole project.  We can also track the # of work points completed each week/month to determine the velocity.  Using this we can calculate the estimated time required to complete all the remaining work points.  I produce a status report every week that includes all these calculations.  Here is a sample of what it looks like:

The beauty of this report is it is extremely easy to generate (when I get some spare time I may write some code that automates the generation of this report by reading from sharepoint directly).  It also provides very useful information, to me, the project stakeholders and upper management.  The % Complete number gives a quick update on the progress being made on the project.  And the Estimated Go Live Date is a useful piece of information to compare against any previously created timelines or deadlines.  Since it's based off of real measurements of progress it makes it much more useful than somebodies (usually mine) wild estimate of when we'll be done.

Do any of you guys do something similar?  Let me know what you think in the comments.

I will be giving a presentation on TDD to the Winnipeg .Net Users Group on Jan 9.  Should be an interesting talk.  Got about 2 hours and I want to go through a quick powerpoint and talk about some of the core concepts and benefits.  Then dive right into developing something simple with TDD.  Not sure what I'm going to use yet for my coding sample.  I might take the classical bowling game example.  But Bowling has been done ad nauseaum...  Sudoku could be fun to do, but last time I tried TDD'ing that I ended up using mock objects and I was hoping to avoid mocks and interaction based testing to keep things simple.  I could also do some boring business-type scenario, transferring funds from one bank account to another or something equally boring.  I think some people might prefer an example like that as it is more applicable to their day-to-day work; but to do any business-type scenario it almost has to include a database which means using mock objects in the tests.  I'll just have to do a test-run of a couple of the ideas and see which one feels best.

Jean-Paul Boodhoo was telling me that he likes to just let the audience pick out a problem.  That sounds like a great idea, can help show how easy it is to apply TDD to any problem by not having a pre-canned demo.  However, then I wouldn't be able to steer clear of some of the more advanced concepts like mocking and inversion of control principles.  If I had 1-2 more hours then I'd just cover mocking also and could tackle pretty much any type of problem as a demo...but in the 2 or so hours I have I just think it would be too rushed.

Anyways, it should be an interesting talk I hope.  Anybody out there who is interested in TDD or in techniques that produce clean, reliable, and easily maintainable code should try and make it out.  Also if anybody has any input as to what they think I should cover (or not cover) or use as a demo problem feel free to contact me (optikal "at" shaw.ca).

In a previous post I mentioned that I had been trying to use Team Build to get a Continuous Integration environment setup.  I recently had the time to return to that task and have managed to get most of what I wanted up and running.  I had to jump through a few hoops to get everything working so I'm going to try and describe what I had to do to make everything work right.

My solution consists of a WinForms UI, ASP.Net Web Service, 2 Class Libraries, Database project (VSTS DBPro CTP7), Testing project, Setup project for UI, and a Setup project for web service.

My testing project contains just over 250 tests.  About 150 of these are what I call "unit tests" that directly invoke methods in one of the DLL's and mock out the database (they run very fast; about 5 secs for all 150).  The other 100 are what I call "acceptance tests" that operate at a higher level, and invoke calls against the web service (I have no tests against the UI project, or directly against the database at this point).  The acceptance tests require the web server to be up and running, and require the database to be wiped out and re-initialized on every test.  It takes about 20-30 mins to run the full suite of 150 acceptance tests.

The first thing I did was create a Team Build using the built-in wizard.  I didn't specify any tests to be run initially.  I had to do a little tinkering around to get the Team Build to work.  There were a few dependencies that I needed to either add to our source control, or install on the build machine.  I had some problems with the setup projects in there, and the database project (the problems with the database project were back in DataDude CTP3 I believe they are fixed in the latest CTP).  What I did to work around these issues was just create a 2nd solution that only contained the WinForms, Class Libraries, and Testing projects.  I had some custom code in the testing project that would auto-deploy the database with the tests, but I had to make sure the build output (the .sql file) from the datbase project was included in source control. 

I did notice that the Team Build didn't seem to like to build my Web Service project.  After a bit of research I found this post by Joel Semeniuk about this issue and how to resolve it.  I had to modify a section of the TfsBuild.proj file to look like this:

<ConfigurationToBuild Include="Debug|Mixed Platforms">
  <FlavorToBuild>Debug</FlavorToBuild>
  <PlatformToBuild>Mixed Platforms</PlatformToBuild>
</ConfigurationToBuild>

Not a big deal once I knew how to do it.  Now the Team Build ran and correctly built all 4 of my projects.

At this point I had a Team Build that I could run on demand from within VS and it appeared to work great.  Now I just had to get it to run my tests as part of the build, and to run automatically on every check-in.

I knew from before that there would be issues with trying to use and maintain Test Lists in our environment since we mostly just use VSTS Dev.  This post by Buck Hodges gave me hope that there was a nice workaround available.  First things first though, I just wanted to get the Team Build running with my tests.  The easiest way to do this was to just fire up Team Suite and create a Test List to use temporarily.  I created a Test List and added all my tests to it.  Next step was to modify my Team Build to run the tests as part of the build.  In my TfsBuild.proj file I had to update the <RunTest> element to True, and add the following section in there:

<MetaDataFile Include="$(SolutionRoot)\ShopFloorTeamBuild\ShopFloorTeamBuild.vsmdi">
  <TestList>BVTList</TestList>
</MetaDataFile>

This tells Team Build to run the Test List BVTList as part of the build.  I fired up my build and noticed that the Unit Tests were all working correctly, but all my acceptance tests were failing.  I had expected this to happen, as there is no Web Server running on the build machine for the tests to hit against.  To fix this I had to add the <AspNetDevelopmentServer> attribute to all of my acceptance tests and add a line of code that performs a little magic on the client-side to redirect the invocation to the proper address.  More info on how this works can be found here.  This is how my code looks for one of my acceptance tests:

<TestMethod()> _
<AspNetDevelopmentServer("MyWebService", "%PathToWebRoot%\MyWebService")> _
Public Sub MyAcceptanceTest()
    Dim Target As MyWebService = New MyWebService
    WebServiceHelper.TryUrlRedirection(Target, Me.TestContext, "MyWebService")
    ...
End Sub

I ran into a little trouble getting this working correctly.  It seemed that I could get it to work either on my local machine, or on the build server, but not both.  It seemed that they were interpreting the %PathToWebRoot% variable differently.  After a little more research I discovered that I can set it up so that it works on the build server, then for the local machine there is a local setting I can change to override where %PathToWebRoot% will point to locally.  It can be overridden in Tools->Options->Test Tools->Test Execution, the Web Application root directory property.

At this point my Team Build is building on demand, and properly executing all of my tests.  I'm getting close now, I can almost taste it.  Time to try and make it run without the test list.

My first attempt was to follow the directions given by Buck Hodges in this post.  I downloaded the task and targets file and set them up on my build server.  Then I modified my TfsBuild.proj file like so:

<!--
<MetaDataFile Include="$(SolutionRoot)\ShopFloorTeamBuild\ShopFloorTeamBuild.vsmdi">
    <TestList>BVTList</TestList>-->
</MetaDataFile>
-->
<TestContainerInOutput Include="ShopFloorTests.dll" />

I tried running the build and all the tests run, however all my acceptance tests are now failing.  After a little investigation I discover that a config file that all the acceptance tests rely on is not getting copied to the tests directory when the tests are run.  I had previously used the .testrunconfig file to specify that this file should be deployed along with the tests.  It appeared that the <TestContainerInOutput> wasn't using my .testrunconfig file (this was confirmed in the comments from Buck's Post).  I was doing some fiddling around with the TfsBuild.proj file, and I had commented out the <TestContainerInOutput> tag, and uncommented the <MetaDataFile> tag but deleted the <TestList> part of it.  It appeared that it was still running all my tests, even though I wasn't specifying what test list to use.  So now my TfsBuild.proj looks like this:

<MetaDataFile Include="$(SolutionRoot)\ShopFloorTeamBuild\ShopFloorTeamBuild.vsmdi">
</MetaDataFile>
<!--<TestContainerInOutput Include="ShopFloorTests.dll" />-->

All the tests run and pass.  I tried removing a test from the project (not touching the vsmdi file) and re-running the build, and sure enough it didn't try to run the deleted test.  I also tried adding a new test, and the build picked that up automatically also (without me needing to modify any test lists).  I'm not sure what I did, but I could swear it never used to work like this.  Maybe it's a result of installing Buck's new task/targets files.

The next step is to get this build running automatically on every check-in.  I followed the approach used by Jeff Atwood here.  That turned out to be pretty straightforward and easy.  Now I just need to setup something so that we can be notified when the build is complete.  After a little reading I discovered the Alerts system built into team foundation server (I've never really noticed it before).  All I needed to do was go Team->Project Alerts...  check off the "A build completes" alert and type in my email.  Every time the team build completes, I now get an email with a link to a build results webpage.

So far so good.  I have my team build running automatically on every check-in.  It compiles all my code and runs all my tests.  It sends me a notification email when it's complete.  That's all the important stuff working.  I still have to play around with getting Code Coverage working in the automated build.  And I would really like to get a build report that just shows me the differences between that build and the previous build as I talked about here.  The most important features of a CI system though are all up and running.

I know the TFS Build team at MS is working hard on the next version, and hopes to make alot of improvements.  I'm hoping that this process will be a little more straightforward in the next version of TFS, and that there will be some added features (such as diff reports between builds, and CI out of the box) that enable us to be more productive and effective at what we do.

Ron Jeffries has an excellent series of articles posted about his experiences TDD'ing the Bowling Game problem.  I read these articles a while back, but I was trying to show them to somebody recently, and it took a while to dig them all up on his site.  So I gathered the links to them all together here for anybody else that might want to read the entire series from start to finish.  He walks you through his thought process as he solves the Bowling Game problem in a few different fashions, TDD'ing each solution.  I especially like the delegate version he produced in one of the articles.

Adventures in C#: The Bowling Game

Adventures in C#: Extending the Procedural Bowling Game

Mining the Bowling Game

Mining the Kicker

Roughing in the Kicker

The Miners Strike

Kick Out the Duplication

The Miners Delegate

New Requirements for Mining Bowling

Continuing the FrameStatus Implementation

I just picked up a new PC on the weekend, and along with it a copy of WoW.  I wanted to play this game the day it came out, but unfortunately my PC at the time was a little underpowered.

The only MMORPG I've played in the past was Dark Age of Camelot (DAOC).  In there I had a lvl 50 (max lvl) tank, and a maxed out armor crafter (really really hard in that game).

I'm on Duskwood - Alliance.  My character right now is a human warrior called Optikal.  I hear that Jeff and John, the owners of geekswithblogs are also on there and have started a guild.  So if either of you guys are reading, I'm looking for a good guild.  I have your characters in my friends list, if I spot you online I'll ping you.  If anybody else reading this is a player, say hi if you see me online.

I've only been playing a couple of days, but I have a few observations about the game.  Inevitably I'm comparing it to DAOC since that is my only experience to date in this genre of games. 

• So far I notice you lvl MUCH quicker in WoW (I'm only lvl 12, this opinion may change).  In DAOC, the amount of xp'ing I've done would probably only get me to lvl 5 or so.
• Grouping doesn't seem nearly as streamlined/popular in WoW.  DAOC had a dedicated interface that you could use to signal you were looking for extra group members and what type.  Then any players in the area could lookup what groups were going and what classes they were looking for.  WoW only appears to have the chat channel for talking about groups, and it looks like it's server wide, so lots of spam and mostly high level groups.  Which brings me to the next point, I haven't yet seen a single group during my hunting.  In DAOC you could pretty much find groups as soon as you hit lvl 7 or so.  Maybe this has something to do with the lvl'ing going by so much quicker in WoW, who knows.
• On a related note, I haven't seen or heard of any low lvl dungeons.  In DAOC I mentioned people started grouping around lvl 7, this is because it was around that level you were capable of hunting in one of the low lvl dungeons.
• A VERY frustrating thing is when fighting multiple mobs, I kill the first one and my tank goes out of attack mode.  EXTREMELY frustrating.  Then I gotta scramble to get the next target and stick my guy back in attack mode.  Sometimes it's not clear for a second or two if I'm in or out of attack mode.  This should just be automatic.  In DAOC you remained in attack mode and automatically started attacking any other enemies that were attacking you.
• DAOC had a /stick command that you could use for melee combat and it worked as you would expect, your character just stayed stuck to the enemy and followed them around.  It could be that without a /stick it makes melee more skillful, which could be a good thing, I guess I just need to get used to it.
• Death is much less of a penalty in WoW.  In DAOC death meant losing something like 5-10% of the XP gained so far this level, and you had resurrection sickness for 5-10 minutes afterward where you were at about half strength. I like the smaller penalty, but I could see it resulting in alot more deaths.  In DAOC if you knew what you were doing you could go for days if not weeks without a death in PvE.  I don't have enough game time yet to know, but with the almost non-existant death penalty, I imagine deaths will be more frequent in WoW. Maybe this is a good thing, I don't know yet.
• I like the talent trees in WoW better than DAOC.  In DAOC you had talent groups like you do in WoW, but you just spend your points on the group, not on specific abilities.  For example, when you hit lvl 40 you get 40 talent points to spend.  If your polearm skill is at lvl 39, then it will cost all 40 points to move it up to lvl 40.  If your shield and crossbow talents are both at lvl 19, then you could spend your 40 points to lvl them both up to 20.  You then get the talent specific abilities at specific levels.  For instance, when you get your polearm talent to lvl's 5, 12, 19, 22, 25, 33, etc you get specific abilities.  I like the ability tree's in WoW better where it gives you more control over what abilities you are interested in.  In DAOC, you had the situation where you may lvl up a talent to 41, just to get the lvl 41 ability even if you didn't care about all the others you would be getting from the earlier levels in that talent.
• The trade skills in WoW seem much more mature than in DAOC.  In DAOC it was lots and lots of boring work to max out your trade skill (and insane amounts of cash).  I was an Armor Crafter in DAOC, and I can't remember how high it went, but I think it was somewhere around 800 that it maxed out.  WoW skills max out at 300, and not only that, the skill points appear MUCH MUCH easier to attain.  Getting a high level crafter in DAOC was literally hell on earth.  The other way to look at that though is that if a maxxed out crafter is so much easier to get in WoW, they may not be as big a deal.  In DAOC there were literally only a handful (about 5-6) armorcrafters on my entire server that could make the high-level armor.  So it was very impressive for somebody to be able to get that high.  But back to what I like about the WoW tradeskills.  Some of the tradeskills allow you to specialize, for instance I'm doing blacksmithing, and later I will be able to specialize to weaponcrafting or armorcrafting, then after that (if I choose weapons) I can specialize further into sword, axes, etc.  And the mechanic where you have to find or buy the high-level plans is really cool.  And also that some of the high-level plans require special drops that people have to do raids to find (I think this is how it works) is even cooler.  In DAOC all the ingredients were store-bought.  This is also because DAOC didn't have the gathering skills like Mining, Skinning, Herbalism, etc.  These are really really cool too, and gets more of the playerbase involved in the crafting game.
• The other thing that is by far the coolest thing that WoW has that DAOC doesn't is the auction house (AH).  What an awesome idea.  It makes the economy SOOO much more fun.  I spent all of last night just buying and selling stuff on the AH, and I multiplied my cash by 10.  Not only that, but it allows people to focus on just the gathering skills and sell their mats to the crafters via the AH.  And crafters can then sell their goods in the AH.  In DAOC the only way crafters could sell their goods was to advertise in the town chat (there was no server wide chat), or in guild chat.  Which meant that you were pretty much only selling while you were logged on and sitting in the main city.  For the lvl 50 crafted goods, people could look up on the game web page which crafters were capable of producing that armor and then send them a /tell, but there was no good way to sell lower level wares.  Again, the Auction House is just awesome in this regard.
• I'm looking forward to trying out the battlegrounds in WoW.  DAOC had battlegrounds, but they were just a big free-for-all.  You stand on the portal to the battleground and every 10 minutes everybody waiting gets teleported over.  It's not instanced (DAOC didn't have instancing for anything), and the battleground is shared between all three realms, with each realm having its own keep they teleport into.  Once you get there it's just a big free-for-all, with the goal of getting as many enemy kills as possible (you got points for PvP kills that you could use to buy special abilities).  You could choose to form a party or go out solo.  There was a keep in the middle of the battleground which was contested, and if you got together a big enough group you could do a raid to try and steal control of it (this was practice for the end-game PvP which was based around this keep stealing idea).  It sounds like the WoW battlegrounds are a little more organized, with special objectives in each one (capture the flag), and they make sure both sides are evenly matched.  Sounds like a blast.

I'm still in the process of experimenting with the various options out there for CI that integrates well with Team System.  I've been playing with CruiseControl.Net and the Team Build system.  While doing this I've got a good grasp on what I would like to see from the ideal CI toolset.

• I want a build to run automatically whenever a developer checks in a changeset.  (I don't want it to run every x minutes/seconds and just hope that it only captures one changeset per build).
• I want the developer that checked in the changeset to be notified when the build completes.
• The most interesting parts of the build report are any compiler errors/warnings, and the pass/fail results of the BVT's.  I want the developer that checked in the changeset to be able to easily see what has changed with regards to errors/warnings and test results since the previous changeset (ONLY what has changed.  eg. if his check-in caused a test to fail I want him to see just the newly failing test; not a list of all 500 tests and which ones are passing/failing, just the one test that his changeset broke).
• All the information from the "diff" report should be displayed on one page (scrolled vertically if necessary).
• The developer should have access to view the full build report in addition to the diff report if he chooses.
• The full build report should include the results from all the tests, the code coverage stats, the compiler logs, and any other relevant data.
• I want to be able to create daily and weekly builds in addition to the CI build.
• I want to be able to view a "diff-only" build report between daily builds, and between the weekly builds.

This is my wish list.  The key item here that I haven't seen done well yet is a tool that can give me a build report that shows only the changes since the last build.  I think this would help my developers tremendously to focus on what is important in a CI environment.

I still have a good deal of learning and experimenting to do with the existing tools (for instance, I'm almost certain I saw somewhere a tool that does CI all within Team System, without using CruiseControl.  I just can't remember where I saw this).  If I can't find pre-existing tools that do what I want, and if I can work up enough motivation, I may take a crack at writing some of this plumbing myself.

I've had a little frustration lately with Team System as a result of not using the complete Team Suite version.  When the licensing for Visual Studio changed last year to the role-based versions and/or Team Suite, we considered the options and decided that VSTS For Developers should be all we need.  We do not do any web or load testing (just unit tests), and we do not have a need for the designers available in the Architect version.

Since then that has held true for the most part.  Except for some important scenarios.  Most of my problems arose when I tried to get a Team Build going.  I managed to get the build itself running.  But then I wanted to enhance it to run our tests as BVT's (Build Verification Tests).  Based on what I'd read, I had believed this was possible.  After all, creating unit tests is part of the capability of VSTS For Developers (you just have a limited amount of control over how you organize them compared to Team Test).  Well it turns out that it's not so easy...in fact it doesn't appear to even be possible.  I installed VSTS for Developers on my build machine, and updated my Team Build to run some tests. But I soon discovered that in order to have the Team Build run some tests you need to specify the test list to use (all I want to do is have it run ALL my tests...why should I have to create a test list for this?).  Well creating test lists is a feature limited to Team Test and not available in VSTS For Developers.  This was confirmed by Eric Jarvi in this forum post: http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=609672&SiteID=1

It turns out that their is no official support for creating a Team Build that includes BVT's if all you own is VSTS For Developers. I believe it may be possible by manually editing the .vsmdi file which is just XML and creating a test list in raw XML.  Rather than mess with that, I decided to just fire up a VPC, install Team Suite Trial, grab a copy of my project from source control, add a test list, then check it all back in.  Then I should just be able to use that test list for my Team Build right?  Sure I'd have to go into my VPC and add in all my new tests every once in a while, and what would I do once the Trial expired, but at least I could get something running right now.

Well sadly not even that would work.  Once I checked it in from the Team Suite edition and tried to run my team build, now it is complaining about all kinds of test related dependencies that aren't available on the build server (because it only has VSTS For Developers).  I played with this for a while, trying to manually edit some of the XML files to remove the dependencies, but after an hour or two of trying I finally got frustrated and backed out my changes in source control.

I still don't have a Team Build running that will run my unit tests.  Before doing anything drastic like abandoning MSTest and Team Build, I have another strategy I want to try.  I think I might be able to modify the MSBuild script used so that it invokes MSTest directly.  Failing that, I'm pretty certain I can use CruiseControl and MSTest.exe as a CruiseControl task.  But in that case I think I might lose the code coverage data.

Anyways, I had somewhere I was meaning to go with this rant.  Most of these difficulties could have been avoided if I was using Team Suite everywhere.  I'm sure many others have experienced similar frustrations or confusion as to what they can and cannot do with the role-based versions of Visual Studio.  My suggestion to Microsoft, is why not do away with the role based versions.  Drop the price of Team Suite to somewhere between what the cost of a role-based version is and Team Suite is today (closer to the role cost than the suite cost).  What you lose in margin you should make up for in volume.  Sure the price to customers will go up a bit, but I think the added cost will be worth eliminating the frustration from trying to get the role versions to do something they weren't meant to do, and the confusion over what it is each one is meant to do and is not meant to do.

I can foresee this problem getting worse once VSTS For DB Pro's comes out.  What if I buy VSTS For DB Pro for my DBA, and VSTS For Developers for my devs.  I have unit tests that my developers write that hit against the database, these unit tests need to deploy the db schema as part of the test.  Will they be able to compile and deploy the database from the dbproj within VSTS For Developers?  It doesn't appear so (at least not until a new version of VSTS For Developers is released).  I understand not allowing them to maintain the schema, you should need VSTS For DB Pro's for that, but the developers will need to be able to deploy that schema in order to run their unit tests.  And to deploy the schema, they should probably also be able to build the dbproj to generate the latest version of the schema.  Even if you ignore the database deployment with unit test issue.  My understanding of MS's vision (and I share in this vision) is that every dev should be running a local instance of the database, and the master version is kept in source control and maintained by the DBA.  Well when the DBA makes updates to the schema how is the developer supposed to deploy those changes to their local database?  My understanding is that VSTS For Developers won't be able to recognize a dbproj at all, let alone compile and deploy it.  Will it be the DBA's responsibility to deploy the database to every developers SQL instance?  How will he know when it is safe to do so?  Will the dev's have to explicitly request the DBA to push out the changes on a workstation by workstation basis when they are ready for a new version?  Not only that (and I may be completely wrong here), but isn't SQL Server Developer Edition restricted to only support local connections.  If so the DBA wouldn't be able to remotely deploy the database to the dev workstations even if he wanted to.

Don't misunderstand me, I am a HUGE fan of the new team system functionality.  If I wasn't I wouldn't get so worked up over these issues.  I'm just starting to feel some frustration with limitations that I don't agree with, and I only see it getting worse down the road if something isn't done to address/improve the issue.  If anybody from the VSTS team is reading, please consider this feedback and think about ways to avoid these problems.  As I said above, I'm all for eliminating the role-based versions and adjusting the pricing appropriately so everybody is either on VS Pro, or VSTS Team Suite.  If that's not an option, then at a minimum consider what common scenario's teams will be trying support when they are using the role-based versions and make sure that enough functionality is included in each version to support those scenarios.  Especially when VSTS For DB Pro's becomes available, I foresee some major hurdles for teams that are using VSTS For Developers for the devs and VSTS For DB Pro's for the DBA's.  To solve the Team Build issues, another possible solution is to provide a special VS install for build servers, that doesn't support development activities, but can support Team Build's that include the features from all the different role versions (build dbproj's, run tests, etc).

We have a problem to solve to do with making subledger/GL entries.  This is a problem that I'm positive has already been solved in every accounting/ERP package out there....I just can't seem to find anybody who knows the answer.

When certain events occurs in our ERP system, we need to make a number of entries to our GL.  A transaction may have multiple debits and credits split up over multiple accounts.  The sum of all the debits must equal the sum of all the credits.  The problem is that these entries must be made to the GL at exactly 2 decimal places of precision (dollars and cents).  The reason that's a problem is because now we have to deal with rounding, and the rounding errors may work out so that the total debits do not equal the total credits.

I'll give an example.  Lets say there was a transaction that credited $1.00 to account A, and had to split the debits evenly over accounts B, C, and D.  After rounding this works out to debits of $0.33 to each of the B-D accounts.  When you sum up the total debits it now only equals $0.99 which is unacceptable.  It gets even trickier when you run into situations where the entries are not split up evenly over the accounts in question.

The simplest way would be to just choose one of the entries at random, and add/subtract the difference due to rounding from it so that the totals even out. Our accountants definately didn't like that idea.  They don't want us to be randomly adding or subtracting from the entry values without some reasonable logic behind it.

My next suggestion was to create a separate account that all rounding error adjustments would go to.  So in the above example, accounts B-D would be debited $0.33, and the extra $0.01 would be debited to this special rounding error account.  Our accountants agreed this would work, but they thought the idea of having a separate account for rounding errors was ludicrous.  They all agreed that in their whole careers as accountants they have never seen that done, and did not wish to do it here.

My next and final suggestion (this was arrived at with the help of one of my developers), was to implement an algorithm that would track the cumulative rounding error as it went through rounding the various entries that comprised the transaction.  Before rounding each entry, it would first add to it the cumulative rounding error.  This appears to work, and is a reasonable solution to the problem.  I am curious if this is a common solution, and if not how this is done in other packages out there that must make ledger entries.

Here is a code example of the algorithm:

This is the algorithm we are currently planning on using.  It is simple, elegant, and appears to work for all situations.  I wrote a quick automated test that loops indefinately through random situations involving a random # of accounts, and a random distribution of values among the credits and debits, and this code appears to pass all situations.  The code I used to test it is below:

I recently started up a project on CodePlex.  The project is to create a software application to help run an offline poker league.  My main reason for creating this was not because I see a huge need out there for an application like this, it's more one of those projects I'm sure we all do to give us a chance to play with some new technology.  In this case it's my chance to get familiar with CodePlex, and do a project from the ground up using TDD and some other agile concepts.  That it can also help with the offline poker league I run is just a bonus.

I'm just in the process of getting up and running, and getting my builds and CruiseControl.Net configured properly.  I have the architecture I'm going to start with all setup, and the skeleton code checked in.

Feel free to check it out at: http://www.codeplex.com/Wiki/View.aspx?ProjectName=PokerLeagueManager

If anybody is interested in helping out with the development let me know.  I was planning on just doing this solo, but if anybody is interested I could always use the help.  It's a great opportunity to experiment with CodePlex, Team Foundation Server, and/or TDD if any of those things interest you.