Build / Release

March 19th: Posts I liked

Manish Agrawal — Tue, 18 Mar 2008 20:16:39 GMT

Inspired by Scott Gu, I also felt the need of blogging the Links/BlogPosts I liked:

** March 19th is the date in "my" context and not that of posts I refer.

Build & Release Automation (it is on top priorities in my list of personal project development)

CI Server: Redefine Your Build Process with Continous Integration: Jay Flowers has written an excellent article on CI and about his tool CI Factory. I think Jay Flower has some similar thinking process I do, CI Factory has almost the same features and has been built upon using CruiseControl.Net as base reference, which even I was doing. The way he has written the title of the article with use of many Upper Case words, also seems to be a matching pattern.

SharePoint (MOSS)

Selling MOSS (The moral of the story): Paul has written an excellent post, he has covered many aspects of proposing a technology and its implementation...

Silverlight

Microsoft Silverlight Streaming Service
IIS 7.0 Bit Rate Throttling Module Released: by Scott Guthrie

.Net

Mono project releases first IDE, Mono 2.0 into beta: by Dana Blankenhorn & Paula Rooney
MonoDevelop Documentation

--Manish

Configuration Environment Architecture

Manish Agrawal — Wed, 26 Sep 2007 13:28:17 GMT

Overview

Based on experience, as well an industry acceptance that development teams are construction driven and are always less focused to Software Process. This has been identified and henceforth the inspiration behind the proposed architecture, as a remedy, by keeping build management beyond developer’s domain.

Modern configuration environment is getting complex due to rising nature of applications to be distributed whereby they are dependent on their functionality on other applications. Such scenarios create enough complications of version control and matching, resulting in dire need of dedicated build management system integrated with the said configuration environment. Henceforth, in the proposed recommendations for Configuration Environment Architecture, Build Manager is an integral entity serving various requirements of configuration control, process and management.

Every file is compiled, linked, and combined into an executable program every day, and the program is then put through a "smoke test", a relatively simple check to see whether the product "smokes" when it runs. Project Server/Manager gets early notification of “smoke”. This simple process produces several significant benefits.

It minimizes integration risk. One of the greatest risks that a team project faces is that, when the different team members combine or "integrate" the code they have been working on separately, the resulting composite code does not work well. Depending on how late in the project the incompatibility is discovered, debugging might take longer than it would have if integration had occurred earlier, program interfaces might have to be changed, or major parts of the system might have to be redesigned and re-implemented. The daily build and smoke test process keeps integration errors small and manageable, and it prevents runaway integration problems.
It reduces the risk of low quality. Related to the risk of unsuccessful or problematic integration is the risk of low quality. By minimally smoke-testing all the code daily, quality problems are prevented from taking control of the project. You bring the system to a known, good state, and then you keep it there. You simply don't allow it to deteriorate to the point where time-consuming quality problems can occur.
It supports easier defect diagnosis. When the product is built and tested every day, it's easy to pinpoint why the product is broken on any given day. If the product worked on Day 17 and is broken on Day 18, something that happened between the two builds broke the product.
It improves morale. Seeing a product work provides an incredible boost to morale. With daily builds, a bit more of the product works every day, and that keeps morale high.

Development Teams

This is a group of roles directly involved in construction and development; constituting of designers, architects, developers etc.

Source Server

Source Server is the source-code repository for version-control. This is the server that is the feeding ground for day-to-day development across projects. Suggestively, this isn’t just a single version-control, but could be a group based on implementation and process management requirements.

Project Server

Project Server is the hosting environment for latest / recent builds submitted by Build Server. This environment is in feedback-loop with Build Manager, whereby each scheduled build is notified of its success status and possibly even hosted on the Project Server to re-affirm the changes being expected in respective build. This ensures early notification of broken integration, and hence early rectification.

Build Server (Build Manager)

This is a Build Management repository, ideal for continuous-integration, smoke testing etc., engaged in continuous and scheduled build management. Notably, this runs on a scheduler which extracts source from Source Server and builds in its environment, mostly resulting in binaries. Resulting builds are versioned and stored on its version-control environment. This not only reduces integration risks of Source Server, but also facilitates early notification to project team of broken integration-builds. Ideally, this is closely controlled and managed by Build Manager.

Build Manager could be a Toolset, Process or a group of role, based on technical feasibility of the environment and platform. The capabilities expected are:

Create, Version and Deploy builds
Notify various managers of build availability and their status.

Initial look at Build Manager might seem like a management over-head, but it goes a long way in segregation of Development, Testing, and Project Management. The intent is to automate most notification and build management tasks. Build deployment on various life-cycle servers could possibly be manually controlled.

There are various advantages of separating Build Server and Source Server in the life-cycle and running scheduled builds:

1. Static Code-Analysis: While generating scheduled builds, source-code could be statically analyzed and reports be made available to Project Team of the issues there-in, reducing code-review and feedback timelines and dramatically enhancing code-quality.

2. Code Documentation: Source-Code documentation can be generated for each corresponding build and versioned along-with. This shall ensure that we have actual mapping of source-code vis-à-vis source documentation. Availability of CHM/HTML docs for each version shall be a boon for developer/project-team reference. Project Manager can henceforth manage/ensure duly documentation.

3. No dependency on Development Team for builds: There shall be no need for Binary builds to be provided by development teams.

4. Focused teams, enhanced productivity: Clear segregation of tasks, and letting development team focus on construction rather than process life-cycle shall enhance productivity.

Test Server & Test Virtual Machines

It is the hosting environment for selective builds hosted by Build Manager, based on build notification received by Project Manager or based on the defined process. This environment is not just a single box to host applications, but also constituted of Test Virtual Machines to support varied configurations required for testing.

Stage Server

Successful builds are notified by Test Manager to Build Manager, which are henceforth deployed on staging based on corresponding process definitions & program manager requirements. Prime subscriber’s of this environment is Program Manager or Customer.

Production Server

Customer or Program Manager approved builds are deployed or provided in Production by Build Manager.

Conclusion

Proper setup of Configuration Environment as per this recommended architecture and with implementation of Build Management System, will not only reduce complexities of application distribution and release management, but will also help in controlling versions of release and will automatically setup an environment which will directly put processes in place thus making releases absolutely reliable and fail-proof.

References: Code Complete, 2^nd ed., Steve McConnell and Article by Sharad Kumar

How To RSS Subversion Change Log

Manish Agrawal — Mon, 21 May 2007 05:13:44 GMT

Brain Ritchie has written an excellent post mentioning the steps to RSS enable your Subversion Change Log. You can check it here.

Thanks Brain and Martin Pittenauer for your contribution.

Issue Trackers

Manish Agrawal — Mon, 02 Apr 2007 18:40:00 GMT

Edd Dumbill has done excellent work by listing out the Issue Trackers with some details here. This is very helpful in deciding to go with which Issue Tracker.

Thanks Edd.. nice list, keep it updating..

How to use Subversion more effectively with proper Repository Structure and better use of Branching/Tagging/Versioning..

Manish Agrawal — Mon, 25 Sep 2006 13:23:00 GMT

In this article, I have assumed that the reader is aware of the basic functionality of Subversion. This article has been written to make the usage of Subversion more efficient and more effective, as not much information is available on this subject in context of Subversion.

Although most of the things are already mentioned in the Subversion Documentation, but going through such vast information in one go becomes something uncommon. So it is good to have small articles targetting a specific feature which can expose the functionality and will get the attention. Without even any extra effort it gets feeded in mind, making understanding of the subject better.

I installed Subversion and started using it, but I started getting problems in maintenance when the code base spreaded and required maintenance of multiple versions.

To find the solution I studied the documentation available, to use Subversion(SVN) more effectively. One of the important things I read was about "What should be the ideal Repository Structure":

So lets understand the options available:

Versioning

In this article I have followed Major.Minor.Build.Revision versioning practice which is very standard, details of it are as under:

In this version number is physically represented as a four-part string with the following format:

<major>.<minor>.<build>.<revision>

For example, version 1.5.60204.0 indicates 1 as the major version, 5 as the minor version, 60204 as the build number, and 0 as the revision number.

Major: Major releases introduce major new technologies and changes that render previous production releases obsolete.
Minor: Minor releases depict feature level enhancements. Addition of features between releases result in incremented minor release.
Build: This is auto-generated number, assigned for each build on a day basis. It has YMMDD format, So Feb 04^th 2006 shall be 60204.
Note: Build number for revisions (read bug-fixes to production releases) shall remain same as originally released in production.
Revision: This is reset to zero, for each new major/minor version released. For all later bug-fixes, patches to releases that reach production, this number shall sequentially increment.

Trunk

Trunk is the main branch of development.

Branch

Isolating changes onto a separate line of development is called Branching. Branches are used to try out new feature without disturbing the main line of development. And as the new feature becomes stable the development branch is merged back into the main branch (trunk).

Tagging is to mark particular revisions (e.g. release version), so that you can recreate a certain build or environment at any time. Tags are used to create a static snapshot of the project at a particular stage. Tagging of the project is mostly done along with the successful build and generally it is done by the automated build process.

Important Note: Subversion itself makes no distinction between tags and branches, but they are there for different purpose. Tags are not normally used for development, Branches must be used for that purpose. So working on a tag revision is not a good idea. So you must remember this as there is nothing to stop you doing this by mistake. However there are few SVN-Client Applications (like TortoiseSVN) which will warn you if you try to commit to a path in the repository which contains /tags/. But I am not sure about other SVN-Clients.

Figure 1: Example of a Subversion Repository Structure (expanded).

Branching or Tagging done by Subversion are just internal links (cheap copies) pointing to a specific tree/revision and thus can be created very quickly and also take up almost no extra space in the repository. If you modify a working copy created from a branch and commit , then all changes go to the new branch and not to the trunk. Only the modifications are stored and the rest remains a cheap copy.

Many times it may happen that you need to make further changes to a release which has been already tagged. The correct way to handle this is to create a new branch from the tag first and commit the branch. Make Changes on this branch and tag the branch for every build with increment in the Revision number.

So finally "What should be the ideal Repository Structure"

Following folder/directories must be created, all at the repository root level:

/trunk
/branches
/tags

How to make the Repository Structure:

Do SVN Checkout to a new folder.
Create the folder /trunk inside the root repository folder.
Create the folder /branches inside the root repository folder.
Create the folder /tags inside the root repository folder.
Copy the project files (which are to be Source Controlled) in the /trunk folder.
Add the files and folder to SVN and do SVN Commit.
Create a Branch with Major and Minor mentioned in the branch name for e.g. "v1_0_X" **
Now you can delete the new folder from your machine (not from SVN) as it is now saved in SVN.
Checkout the newly created branch. Now you can start working on this.

If you need to make further changes to a release which has been already tagged, following steps must be followed:

Checkout the tagged revision to a new folder.
Create a new Branch from the checkedout project.
Switch the working copy to new Branch (or in simple words Delete the just created new folder and checkout the new branch to a new folder.)

For e.g. Suppose you need to make further changes to a release tagged "v1_0_60924_0". Steps will be as under:

Checkout the tag "v1_0_60924_0" to a new folder say "C:\HelloWorld_temp".
Create a new branch "v1_0_60924_X" *** by giving the command from within the above folder.
Switch the working copy:
- Deleting the "C:\HelloWorld_temp" folder.
- Checkout the branch "v1_0_60924_0" to a folder "C:\Projects\HelloWorld".

** Branch name v1_0_X dissected: "v" represents the word "Version", "1" is Major, "0" is Minor and "X" is to indicate that along with the changes (or during the Build process) Tags will be created with dynamic Build and Revision numbers, for e.g. "v1_0_60923_0" here 60923 is the Build Number which represents the date of build (6 is the year 2006, 09 is the Month and 23 is the Date), after Build Number there comes the Revision Number which will get incremented with every next build during the day.

*** In case of already tagged release Branch name used is "v1_0_60924_X" because for further builds on this Branch only revision number will get incremented. Mostly this happens when a Build is released or finalized to be released, then for bug-fixes and patches to be release the Build number is freezed and only Revision number changes.

How To: Install the Subversion SVN (HTTP Module/Server/Client) on Windows

Manish Agrawal — Thu, 15 Jun 2006 02:22:00 GMT

Download the binaries

For Server machine:

Go to the Apache download page and download the version (apache_2.0.54-win32-x86-no_ssl.msi) of Apache web server. I tried downloading some latest versions also but faced few issues in installation, so I continued with version 2.0.54.

Go to the Svn1ClickSetup download page and download the latest version. As of the writing of this article it is 1.3.2.

For Client machine/s:

It will be sufficient to just install Tortoise SVN Client, it can be downloaded from TortoiseSVN Page. As of the writing of this article latest version was 1.3.4

Set up Subversion on Server

Run the Svn1ClickSetup executable (Svn1ClickSetup-1.3.2.exe) downloaded in the previous step. All the installation locations mentioned in this article are assumed to be default locations.

Note: Svn1ClickSetup installs a Subversion service, if you don’t want to be running this service, open a command prompt, and run the following command:

svnservice -remove

Set up TortoiseSVN on Client

Run the TortoiseSVN Setup executable (TortoiseSVN-1.3.4.6692-svn-1.3.2.msi) downloaded in the previous step.

Note: If you are working on VS.Net with Web Projects, it is recommended to select ASP.Net Hack on the Custom Setup step of the installation wizard. Although I have not explored consequences of not selecting it but it is hardly of any harm, as it not even takes 1 kb of your hard disk space.

Install Apache

Run the Apache installer (apache_2.0.54-win32-x86-no_ssl.msi) that you downloaded. It is pretty simple to install.

Once the installation finishes, launch a web browser, and browse to http://localhost. If you see a test page from Apache, the server is installed correctly.

Note: If port 80 is already occupied, you will have to stop the Webserver on which it is running or you will have to stop that WebSite. As Apache installation wizard gives not enough options of setting port while installation.

Later you can change the default port (which I recommend) in the httpd.conf file located in C:\Program Files\Apache Group\Apache2\conf folder.

You will have to make changes at two places, firstly search for word Listen and replace 80 with your desired port number for example 8181

Listen 8181

Secondly search for ServerName and replace :80 with your desired port number.

ServerName yourdomain.com:8181

You will have to restart Apache and now you can launch a web browser, and browse to http://localhost:8181/

Note: Now don’t forget to restart the Webserver or WebSite, which you mayhave stopped.

Set Up HTTP Access

The first thing we need to do is make sure that Apache loads the WebDav modules. The Apache config file (httpd.conf) is located in the C:\Program Files\Apache Group\Apache2\ conf directory. Launch a text editor, and open the Apache config file. You will want to keep this file open for the rest of the installation.

Copy the Subversion HTTP modules:

Program Files\Subversion\bin\mod_dav_svn.so

Program Files\Subversion\bin\mod_authz_svn.so

into the Apache modules directory (Program Files\Apache Group\Apache2\Modules).

In the config file, look for a section of lines that all start with LoadModule. At the end of that section, add the following lines:

LoadModule dav_module modules/mod_dav.so

LoadModule dav_svn_module modules/mod_dav_svn.so

LoadModule authz_svn_module modules/mod_authz_svn.so

At this point, restart the Apache2 service (Control Panel > Administrative Tools > Services). You will probably want to keep the Services window open (we will be restarting Apache several more times). If it restarts with no errors, Apache is now loading the SVN module.

Back in the Apache config file, add the following to the end of the file:

  DAV svn

  SVNPath "C:\svnrepos"

Restart Apache2 again. You can now access your repository via HTTP.

Configuring Authentication

?ote: There are several different ways to configure authorization for Subversion, but in this article only basic authorization is mentioned. Basic authorization sends the passwords in near plain-text, so if you are concerned with password snooping, you will have to use a different authorization scheme.

Now, we can browse our repository via HTTP. Not only that, but we can also check-out, modify, and commit changes. However, none of these operations require any type of authorization. Not exactly what we want, especially if we are going to have multiple users.

To enable authorization, we need to tell Apache who the authorized users are. To do this, we give Apache a file with a list of the authorized users and their passwords. Apache makes this job easy by providing a utility to mange the user file, htpasswd.

Let’s go ahead and create our user file with the users Tom and Jerry. We are going to put our file in the Apache conf directory, so open a command prompt to that directory (C:\Program Files\Apache Group\Apache2\conf). Type the following command, and then enter a password for Tom.

C:\Program Files\Apache Group\Apache2\bin\htpasswd -cm svn-auth-file Tom

Now type the following command to add Jerry to the file.

C:\Program Files\Apache Group\Apache2\bin\htpasswd -m svn-auth-file Jerry

Note: There is difference between these two commands. The first command passes in the -c command, which tells htpasswd to create the file. The second time, the file already exists, so just -m command is enough. The second command can be used to add additional users to the file. In the svn-auth-file entries for both Tom and Jerry can be seen.

Now that authorization file is created, some Apache configuration is also required. Go back to the Location tag that we added to the Apache config file. And add some more lines to it so that it looks like this

  DAV svn

  SVNPath "C:\svnrepos"

  AuthType Basic

  AuthName "Subversion repository"

  AuthUserFile "C:\Program Files\Apache Group\Apache2\conf\svn-auth-file"

  Require valid-user

After restarting Apache2 once again, browse to your repository again. This time, you will be prompted for a user name and password. Cool, we have authorization working. However, if you don’t want to be prompted for a user name unless modifying the repository, you are not quite finished yet.

Thankfully, only requiring authorization for modifications is really easy. All you have to do is add a couple more lines to the location tag so that it looks like this

  DAV svn

  SVNPath "C:\svnrepos"

  AuthType Basic

  AuthName "Subversion repository"

  AuthUserFile "C:\Program Files\Apache Group\Apache2\conf\svn-auth-file"

    Require valid-user

One last restart of Apache2 and you now have the Subversion HTTP module installed, configured, and ready to control our revisions.

Reference: http://blog.briankohrs.com/2005/09/20/guide-to-installing-the-subverison-http-module-on-windows/#comments

I have modified Brian Kohr’s article as I felt it was missing few things. I hope it will be of help and will be taken in the right spirit.

Problem with Digitally Signing Assemblies...

Manish Agrawal — Tue, 21 Feb 2006 00:54:00 GMT

Narendra has written a good post on resolving the problem related to Digitally Signing of Assemblies is .Net 2.0
(url : http://geekswithblogs.net/narent/archive/2006/02/15/69536.aspx)

The problem is:

1. To digitally sign assemblies “AssemblyIncludeSigningInformation” must be set to true, but once it is set to true
it becomes mandatory to write “AssemblyDelaySign” attribute in AssemblyInfo.cs file.
2. There is a logical error in the code (of AssemblyInfoTask.cs), which is mentioned by Narendra in the post.

For any user it will become a problem in case assembly is to be digitally signed.
As no where it is mentioned that DelaySign Attribute must be added in the AssemblyInfo.cs file and also how the
AssemblyDelaySign value must be given (i.e. it should not be left blank).
For e.g. this is an incorrect entry [assembly: AssemblyDelaySign()]

If the incorrect entry is given, it will get converted to [assembly: AssemblyDelaySign(“false”)]
which will further lead to error in compilation.

Where as Correct Entry must be: [assembly: AssemblyDelaySign(false)]