Continuous Integration

Hyper-V for Developers - presentation from Edge UG 16 September 2009

Liam Westley — Thu, 17 Sep 2009 14:29:04 GMT

Thanks to Ray and Gavin for organising another top EdgeUG meeting in Microsoft Cardinal Place and for everyone who attended my talk on Hyper-V for Developers. The audience had to deal with a huge amount of technical information that I crammed into the hour - hopefully it didn't put them off their pizza.

Fortunately the Stack Overflow master and author of C# in Depth, Jon Skeet, was on hand after the break for a interesting glimpse into code contracts coming with VS2010 and C# 4.0.

For those who weren't there last night, the presentation provides an overview of Hyper-V Server 2008 and Windows Server 2008 running Hyper-V, tools that you can install to help you out, how Snapshots work and finally automation of Hyper-V using the PowerShell library for Hyper-V as might be used in CI (continous integration) server.

I've got two version of the slidedeck, one in PowerPoint, hyperv4dev-ppt.zip, and a PDF version complete with speaker notes that contains all the interesting stuff including the PowerShell script, hyperv4dev-pdf.zip.

Thanks again to the guys from EdgeUG and all those who joined us down the pub afterwards for a beer and a chat.

Liam

P.S. Jon mentioned that Microsoft has decided that static checking of Code Contracts should be limited to the Team System versions of Visual Studio 2010. Here is a link to the Microsoft Connect request to reverse this blinkered approach and place static checking into Visual Studio Professional edition;

https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=481327

Do please vote on t his topic and let Microsoft know what you think, hopefully it 's not too late to reverse this marketing decision.

Hyper-V for Developers - presentation from DevEvening 13 August 2009

Liam Westley — Fri, 14 Aug 2009 15:01:13 GMT

Thanks to Alex and Matt for organising another top DevEvening user group in Woking and for everyone who attended my talk on Hyper-V for Developers. Apologies for over running and hope it didn't stop you from enjoying your meals. I think I achieved a good warm up for Sebastien Lambla's talk on best practices in MVC.

For those who weren't there last night, the presentation provides an overview of Hyper-V Server 2008 and Windows Server 2008 running Hyper-V, tools that you can install to help you out, how Snapshots work and finally automation of Hyper-V using the PowerShell library for Hyper-V.

I've got two version of the slidedeck, one in PowerPoint, hyperv4dev-ppt.zip, and a PDF version complete with speaker notes that contains all the interesting stuff including the PowerShell script, hyperv4dev-pdf.zip.

Thanks again to the guys in Woking, and for Sebastien Lambla for helping me polish off a bottle of nice wine on the way back to London.

NxtGenUG Oxford - CI Factory talk

Liam Westley — Thu, 07 Aug 2008 21:02:50 GMT

Thanks to all who attended the NxtGenUG Oxford meeting last night, always good to see a different user group in operation. Apologies for not achieving the promised 20 minutes to create a CI server from a completely clean Win2003 server installation - but 26 minutes isn't too shoddy.

As promised you can download the slidedeck for my CI Factory talk from here. Make sure you view it with the speaker notes as they give all the background information. On the 'Demonstration' slide the notes include the exact steps and installation list for creating that CI server in 26 minutes.

And finally, thanks to Barry Dorrans for ferrying me to and from Didcot Parkway station (and providing a Hot Wax chilli seedling) as well as Research Machines for providing the factilities for the evening.

Liam

P.S. In the demonstration I used a full .NET 3.5 redistributable package to install the .NET SDK without requiring an internet connection. Thanks to the articles on Aaron Stebner's WebLog and Bret Grinslade blog I idenitified the correct package to download. The full package also arrived this week on MSDN CD 2426.29 (grey).

London .NET User Group - CI Factory talk

Liam Westley — Fri, 25 Jul 2008 21:29:24 GMT

Thanks to all who attended the London .NET User Group meeting last night, a great turnout and lovely weather for slurping beers in the streets of Clerkenwell afterwards.

As promised you can download the slidedesk for my CI Factory grok talk from here, and if yoiu want an extended version I'm presenting at NxtGenUG in Oxford on Wednesday 6th August.

In the one hour or so I'll hopefully have time to actually create a SubVersion repository, edit the CI Factory setup file to create a new build server and demonstrate it automatically building an app checked in from Visual Studio 2008.

And finallyl, thanks to Ian Cooper and Skillsmatter.com for providing the factilities for yesterday evening.

DDD6 - 24th November - You have until 19th October to get your vote in

Liam Westley — Mon, 15 Oct 2007 11:04:14 GMT

Voting now open on DDD6 here, http://www.developerday.co.uk/ddd/votesessions.asp, so get your 10 votes in to make sure it's the conference you want it to be.

Those who are eagle eyed will note that my bribes to the organisers were obviously well received as my yap about CIFactory has appeared at the top of the voting table .....

Grok talk at Fest07 - 23rd May 2007

Liam Westley — Mon, 21 May 2007 10:50:06 GMT

I will be presenting a grok talk at Fest07 at Microsoft campus in Reading; 'Continuous Integration for newbies made easy - A quick guide to CI Factory'.

Those who have read my posts on CI Factory installation will be familiar with the subject, but feel free to come along and say hello if your attending. I'll have my build server with me - an external notebook drive with VirtualPC image and I'll be giving a brief overview of how CI Factory got my build server up and running in a couple of days, with a comprehensive set of code analysis tools.

The Fest07 conference is organised by the NxtGenUG (Next Generation User Group).

UPDATE: Presentation now available here (PowerPoint 2003 and Acrobat PDF versions).

Technorati tags: ci factory, Continuous Integration

CI Factory - my final setup

Liam Westley — Tue, 24 Apr 2007 10:08:39 GMT

Here's the culmination of my research into CI Factory and how I'm not using it to develop my open source RapidDB libraries. This is a fairly personal setup, as it only has a single developer (me), two development machines (currently an office desktop PC and a roaming laptop) and a build server which is not publicly accessible via the Internet. An MSDN and Microsoft Partner Action Pack subscription are busy covering my software license requirements.

Summary:

Development PC 1 : Desktop, XP Pro SP2, Program Files on D: drive, C:\Projects for CI Factory tree.
Development PC 2 : Laptop, XP Pro SP2, Program Files on C: drive, C:\Projects for CI Factory tree.
Build Server : Virtual PC 2004 image, XP Pro SP2, C:\Projects for CI Factory tree.
Source code repository : www.hosted-projects.com.

By using Microsoft Virtual PC 2004 (or VPC), which is a free download now, I have the VPC disk image (6.3Gb currently) which is on a portable laptop hard drive, running over USB 2.0. When in the office this connects to my desktop, when out of the office this connects to my laptop. That way, I always have the build server available, where ever I am working. And when my build server finally settles down and I require external access, I can easily install it under Virtual Server on my main Windows Server 2003 box and setup suitable port forwarding on my broadband connection.

Both development PCs and the build server have Microsoft SQL Server 2000 Developer and Microsoft SQL Server 2005 Developer installed. To aid with unit testing, I have created aliases to the servers on all machines as NUnit-Sql2000 and NUnit-Sql2005. This ensures that whatever machine name and instance name I might have used when installing SQL Server, unit tests can use the same alias names across all machines (and please don't flame me that unit tests should not really include database access, I have my reasons).

Build server o/s and tools installation

Windows XP Professional SP2
.NET framework 1.1, .NET framework 2.0, .NET framework 3.0
Microsoft SQL Server 2000 Developer Edition, Microsoft SQL Server 2005 Developer Edition
NCover v1.5.5 (beta)
SubVersion 1.4.3, TortoiseSVN 1.4.3
.NET Framework 1.1 SDK, .NET Framework 2.0 SDK [blog update, 6 June 2007]

CI Factory installation on VPC build server image

.\Install Scripts\Arguments.xml;

Change ProjectName; testproject.
Change the portname number (really important for multiple installations!)
Email details; VPC-Build, vpc.build@yourdomain.co.uk
Developer lists; your.name@yourdomain.co.uk
SVN URI Root; https://svn1.hosted-projects.com/youraccountname
SVN URI ProjectName, Shared Repo; ProjectName" value="${SVN.URI.Root}/${ProjectName}"
SVN.WebRepoURL; https://svn1.hosted-projects.com/youraccountname/${ProjectName}
SVN.Credentials.SafeStorage; true
SVN.Username / SVN.Password; <set to your username/password>

\Packages;

Install NCover 1.5.5 Beta to prog files; C:\Program Files\NCover
Install SubVersion 1.4.3 to prog files; C:\Program Files\Subversion
(optional) Install TortoiseSVN 1.4.3 to prog files; C:\Program Files\TortoiseSVN

CI Factory Installation

Create repository/project for testproject in SubVersion, edit ACL list to include user permissions RW.
Run C:\Tools\CI Factory\run.bat to install.
After install Visual Studio will not start as it doesn't exist on the build server; instead use
   Notepad to edit c:\Projects\testproject\Current\Build\Main.build.xml
   --> look for warnings about first call/last call and move (as in Jay Flower screencast)
   --> (SourceModificationReport.PublishOldSource to first item)
   --> (SourceModificationReport.PublishNewSource to last item)
Commit, Main.build.xml to SubVersion.
Run C:\Projects\testproject\Current\Build\Packages\CSDiff\bin\CSDiff.exe once to store path to CSDiff in registry (used in later scripts).

Post installation configuration; c:\Projects\testproject\Current\Build\ccnetproject.xml

Locate <!ENTITY email line (at top of file)
Edit the from attribute to a suitable 'from' address
Edit the mailhost attribute to by your SMTP server
For authenticated logins add the mailhostUsername and mailhostPassword attributes and enter the login and password used to send e-mails.

Afterwards, it should look something like;
<!ENTITY email '<email from="testproject.build@yourdomain.co.uk" mailhost="mail.yourdomain.co.uk" mailhostUsername="testproject.build@yourdomain.co.uk" mailhostPassword="testprojectbuild" includeDetails="true">

Developer PC 2 - C:\Program Files (99% of users)

You can just check out the tree from SubVersion to C:\Projects\testproject and run C:\Projects\testproject\Current\Build\OpenSolution.bat to load the project into Visual Studio 2005. If you want to run unit tests you will need to have installed the appropriate tools (NUnit for myself, or feel free to use MbUnit which comes with CI Factory).

Developer PC 1 - D:\Program Files

Check out the tree from SubVersion to C:\Projects\testproject.

Copy C:\Projects\testproject\Current\Build\OpenSolution.bat to C:\Projects\testproject\Current\Build\OpenSolutionVSOnD.bat
Edit OpenSolutionVSOnD.bat in Notepad, and edit the path C:\Program Files\Microsoft Visual Studio 8\VC\vcvarsall.bat to be on the D: drive.
Repeat with C:\Projects\testproject\Current\Product\OpenSolution.bat.

Adding your projects

Now you continue with the normal CI Factory process of adding your existing projects into C:\projects\testproject\Current\Product\Production as detailed in the Jay Flowers/CI Factory screen cast.

Conclusion

For those thinking that all this sounds like a whole heap of work to 'reproduce' your existing setup, this is really is not the case. I cannot tell you the relief to have CruiseControl.NET setup for me automatically and have it branching versions seamlessly in the background. It also encourages you down the unit testing path which is not a bad thing at all (although I may be breaking it with database reliant testing).

I have also been impressed by the range of default packages installed by CI Factory. This provides an immediate wake up call to how you might improve the quality of your software by using automated tools, instead of just hoping you have written reliable code that you have tested properly. I have yet to use all the default packages properly, but then, even programmers I know who have in depth experience of Test Driven Development (TDD) and CI have found new tools that CI Factory includes, such as Simian.

Technorati tags: ci factory, Continuous Integration

Update log
6 June 2007 : Updated to include installation of .NET Framework SDK to prevent Unable to load DLL 'svn_client-1' error in build log (as in CI Factory FAQ, here).

CI Factory with NUnit (rather than MbUnit)

Liam Westley — Mon, 16 Apr 2007 12:24:23 GMT

I decided to add some unit tests into my first CI Factory project. First I used a reference to MbUnit.Framework.dll in my test project, this is the unit test framework which ships with CI Factory, and all worked fine.

No disrespect to MbUnit, but I decided I really wanted to use NUnit so I can utilise the larger amount of documentation and community knowledge that exists for NUnit (even though it might not be as flexible as MbUnit). So I added NUnit 2.2.8 (.NET 2.0 version) underneath my CI Factory Third Party folder and flipped the test project over to use NUnit with a reference to NUnit.Framework.dll instead of MbUnit.Framework.dll.

Cruise Control blew up with an error in the NAnt Output log;

External Program Failed: C:\Program Files\NCover\NCover.Console.exe (return code was 1)

Oops!! I know the issue is within the file Build/Packages/NCover/NCover.Target.xml, probably leading to Build/Packages/DotNetUnitTest/UnitTest.Target.xml but I couldn't find anything obvious to edit.

In the end the Solution was horribly simple and I let out a loud DOH!. I just left the reference to MbUnit.Framework.dll in my test project, in addition to the NUnit.Framework.dll reference which actually performs the unit tests.

I can now run the NUnit GUI to run my tests on my development machine. If they pass, I check in, Cruise Control builds the project and happily runs my unit tests in automated mode.

Happy, happy, joy, joy

P.S. I did muse that it could be the lack of an NUnit.Tasks.dll for NAnt to hook into which might be the issue, but I'm no expert so this is a bit of a wild stab in the dark.

Technorati tags: ci factory, Continuous Integration

SubVersion - make sure SVN.EXE and Tortoise are kept in sync when using automated tools

Liam Westley — Wed, 11 Apr 2007 02:29:00 GMT

A minor glitch while getting CI Factory up and running.

I discovered how important it is to make sure that you keep your version of SVN.EXE (command line SubVersion) in step with that of TortoiseSVN. As TortoiseSVN informs you of upgrades, I'd happily updated it to v1.4.x automatically, leaving my SVN.EXE at 1.3.x.

The gotcha was that the installation of CI Factory was fine - it used the older v1.3.x client to create the project tree. After I had used TortoiseSVN to add some projects, it was only when I forced a rebuild by CruiseControl.NET that I saw errors in the CC.NET interface detailing the SubVersion mismatch.

The issue was that you cannot use older SubVersion clients to update a local tree created by a different SubVersion client. Quite a reasonable requirement and a terrible 'DOH !' moment on my part. A quick update of my command line SubVersion tools and all was well in CC.NET.

CI Factory - getting it working with your development tools in D:\Program Files

Liam Westley — Wed, 04 Apr 2007 05:06:00 GMT

Before we dive in make sure you read my introduction post here, and that you have followed the screen cast on how to setup CI Factory. My notes will basically follow that procedure and highlight the differences when you have development tools on your D: drive.

This is an early list, but hopefully it should give people pointers on how to get going quickly;

I'm assuming that your development PC already has Visual Studio 2005, and SubVersion 1.4.3. In addition I had to install NCover 1.5.5 beta and download the latest CIFactory beta (I used 0.8.0.121) which I unzipped to C:\Tools\CI Factory. I have hosted SubVersion at hosted-projects.com for $15 a month (note: my host area has been changed to xxxxxx on this server for obvious reasons).

Once unzipped, look at C:\Tools\CI Factory;

.\Install Scripts\Arguments.xml (standard edits)

1. Change ProjectName; testproject.
2. Change the port number if you want multiple installations.
3. Edit the email details and developer list e-mails.

4. SVN.URI.Root; https://svn1.hosted-projects.com/xxxxxx
5. SVN.URI.ProjectName, Shared Repo; ProjectName" value="${SVN.URI.Root}/${ProjectName}"

6. SVN.WebRepoURL; https://svn1.hosted-projects.com/xxxxxx/${ProjectName}
7. SVN.Credentials.SafeStorage; true
8. SVN.Username / SVN.Password; <set to known login for hosted-projects.com>

.\Packages (D: drive specific)

1. Install NCover 1.5.5 Beta to program files; D:\Program Files\NCover
2. Edit .\NCover\NCover.Properties.xml; Ncover.ExePath to D:\Program Files\NCover\NCover.Console.exe
3. Edit .\NCover\Install\Properties.xml; Ncover.ExePath to D:\Program Files\NCover\NCover.Console.exe
4. Edit .\MSBuild\Install; %Compile.DevEnv.vsvars32bat% to D:\Program Files\Microsoft Visual Studio 8\VC\vcvarsall.bat
5. Edit .\VS.NETCompile\Install; %Compile.DevEnv.vsvars32bat% to D:\Program Files\Microsoft Visual Studio 8\VC\vcvarsall.bat

.\Power Tools (D: drive specific)

1. OpenSolution.bat; edit C:\Program Files\Microsoft Visual Studio 8\VC\vcvarsall.bat to be on D: drive.
2. Still in OpenSolution.bat; edit C:\Program Files\Subversion\bin to D:\Program Files\Subversion\bin

.\Default\Build (D: drive specific)

1. CCNETServer.bat; edit C:\Program Files\Subversion\bin to D:\Program Files\Subversion\bin
2. OpenSolution.bat; edit C:\Program Files\Microsoft Visual Studio 8\VC\vcvarsall.bat to C:\Program Files\Microsoft Visual Studio 8\VC\vcvarsall.bat.

To prevent NDepends build errors
Copy D:\WINDOWS\FONTS to C:\WINDOWS\FONTS to prevent error in CCNet Build Report erroring on missing Arial font.

>> INSTALLATION

1. Create repository/project in SubVersion, edit the ACL/permissions list to include user permissions RW for the login you configured.
2. Run C:\Tools\CI Factory\run.bat to install CIFactory components (I left them with the default modules).
3. After install Visual Studio fires up; edit c:\Projects\testproject\Current\Build\Main.build.xml
   --> look for warnings about first call/last call and move (as in the CIFactory screen cast)
   --> (SourceModificationReport.PublishOldSource to first item)
   --> (SourceModificationReport.PublishNewSource to last item)
4. Commit, Main.build.xml to SVN.
5. Run CSDIff once to store path to CSDiff in registry (this is used in later scripts).

>> ADD INITIAL PROJECTS (to C:\Projects\testproject\Current\Product\Production)

1. Run, C:\Projects\testproject\Current\Product\OpenSolution.bat; this should load Visual Studio 2005.
2. Copy your existing projects to C:\Projects\testproject\Current\Product\Production (use TortoiseSVN->Export to get a clean tree with no .svn folders)
3. Add your projects in Visual Studio, check it builds.
4. Add your projects in C:\Projects\testproject\Current\Product\Production, to SVN (remove/exclude the bin/obj directories).
5. Commit your additions, and the changes to the main solution (new projects) to SVN.
6. CruiseControl.NET should now pick up changes and compile your new solution.

Miscellaneous notes

The resource usage for an 'empty' CIFactory install with default modules but none of your own projects came in at 14Mb with SVN on hosted-projects.com (peaked at 17Mb during commit). Important to note if you have usage limits.

The other wrinkle I had was a SubVersion certificate issue; hosted-projects.com are still using a certificate based on the URL http://www.hosted-projects.com/ which was created prior to their configuring multiple servers. My repositories are now on svn1.hosted-projects.com, however they still use the old certificate for encrypting content. This doesn't bother me but it can bother svn.exe, the command line tool, especially when used in an automated mode.

To fix this run SVN.EXE at the command line to your repository;
     svn list https://svn1.hosted-projects.com/xxxxxx/cifactory --username xxxx --password xxxx
You will be asked to accept the certificate, if you choose permanently then the CIFactory and CCNET scripts will work without a problem.