I'm still in the process of experimenting with the various options out there for CI that integrates well with Team System. I've been playing with CruiseControl.Net and the Team Build system. While doing this I've got a good grasp on what I would like to see from the ideal CI toolset.
- I want a build to run automatically whenever a developer checks in a changeset. (I don't want it to run every x minutes/seconds and just hope that it only captures one changeset per build).
- I want the developer that checked in the changeset to be notified when the build completes.
- The most interesting parts of the build report are any compiler errors/warnings, and the pass/fail results of the BVT's. I want the developer that checked in the changeset to be able to easily see what has changed with regards to errors/warnings and test results since the previous changeset (ONLY what has changed. eg. if his check-in caused a test to fail I want him to see just the newly failing test; not a list of all 500 tests and which ones are passing/failing, just the one test that his changeset broke).
- All the information from the "diff" report should be displayed on one page (scrolled vertically if necessary).
- The developer should have access to view the full build report in addition to the diff report if he chooses.
- The full build report should include the results from all the tests, the code coverage stats, the compiler logs, and any other relevant data.
- I want to be able to create daily and weekly builds in addition to the CI build.
- I want to be able to view a "diff-only" build report between daily builds, and between the weekly builds.
This is my wish list. The key item here that I haven't seen done well yet is a tool that can give me a build report that shows only the changes since the last build. I think this would help my developers tremendously to focus on what is important in a CI environment.
I still have a good deal of learning and experimenting to do with the existing tools (for instance, I'm almost certain I saw somewhere a tool that does CI all within Team System, without using CruiseControl. I just can't remember where I saw this). If I can't find pre-existing tools that do what I want, and if I can work up enough motivation, I may take a crack at writing some of this plumbing myself.