Setup

Uninstall a component during minor upgrade

Vagmi Mudumbai — Sun, 11 Jun 2006 11:42:00 GMT

It is bound to happen. You will have a developer come up to you one day and say, "Remember the DLL file that we shipped in the previous release, we have consolidated its functionality into the core library. We need to remove that library in the latest release." You as a Windows Installer developer would then be frustrated after finding out that you cannot remove components during a minor upgrade and you will have to manage with all the ugliness of a major upgrade. I will write about the ugliness of a major upgrade in some other blog.

So if you really have to remove the DLL, you need not have to go through the trouble of major upgrade. We can actually uninstall components during a minor upgrade. But the catch is that you should not remove the component from the existing MSI. My colleague Kajal Biswas calls this process as "Puncturing the component". Let me detail the process of puncturing the component.

Consider the component abc.dll which has to be removed.
Collect the Registry, CLSID and the COM information if you have dynamically acquired them, you will have to fill them in manually later. This is usually the reason why I do not use COM Extract at Build. I usually use 'Extract COM Information' feature of InstallShield DevStudio. This would ensure that I can find out the exact registry entries and COM Settings. If you had used the WiX Toolset, it is pretty straight forward. You will just have to go to the next step and you can ignore this completely.
Replace abc.dll file with a zero byte file with the same name.
Set the transitive bit of the component. If you are using InstallShield DevStudio, simply set the 'Reevaluate Condition' field in the component property to 'Yes'.
Replace any existing condition on the component with a condition that evaluates to false. As a habit, I and my colleagues enter the paradoxical '1=0' as the component condition. You can also enter something meaningful like 'GoToHell=666' or something like that.

During a minor upgrade, if the feature associated with the component is reinstalled, the trasitive bit with also force the state of the component to be reevaluated. And since the component is present and the condition is false, the component will be uninstalled during a minor upgrade. It is hack. But it works.

The transitive bit gotcha!

Vagmi Mudumbai — Sun, 02 Apr 2006 11:09:00 GMT

Windows Installer's transitive bit is a really cool thing. It allows you to author your components in such a way that it would install if component's condition is true and uninstall if it is false even during upgrades. Well almost. Refer the documentation for more information. So this can be used to get rid of pesky components that you need to uninstall during a minor upgrade. Now we thought this stuff was really neat and applied it to all the components. We had our justifications that this setup will always be done by admins and will always be per machine. None of the components wrote anything off to HKEY_CURRENT_USER and we were pretty much safe in assuming that it would not be the case in the future either. So we turned on the transitive bit for all our components. The software behaved exceptionally well. The maintenance was driven via a external wrapper which always passed the REINSTALL parameter and hence everything worked very well.

This allowed our MSI to behave as a state machine. You could simply throw in a bunch of properties and set its state and it would just install the ones that are required. This basically solves the problem of setup types and you do not have to write custom actions to set the feature request states.

We had tested most of the scenarios on different operating systems and it seemed to function just fine. We had it going through the last round of validation when we got a call from one of the landscape administrators that the product had uninstalled itself. Windows Installer had triggered a repair after the application of a hotfix by one of the developers and had completely uninstalled the application. The reason being that the Windows Installer repair operation also re-evaluates the component conditions and decides that they should not exist on the machine. The problem rested in the fact that even though we did persist the properties to be retrieved later, but that was entirely upto the bootstrapper to do it as the bootstrapper was the single entity that would do all the MSI related maintenance activities. So when the MSI repair started, which of course is as good as a quick reinstall, it ran through the entire InstallExecuteSequence table and found that most of the components were not supposed to be on the machine and had them removed. The solution was fairly simple. We had to persist all the properties and its values in the registry and we had to author AppSearch entries for all the properties and do a sort of quick check for the settings and swap the values with the actual ones during such a repair operation.

So if you plan to use the MSI transitive bit for your components, be very careful. You have been warned.

Multi Platform versus Cross Platform

Vagmi Mudumbai — Fri, 07 Oct 2005 20:22:00 GMT

I have lately been pondering over methods to deploy cross platform products. I investigated certain options like InstallShield's Universal Project and ZeroG's InstallAnywhere. But to tell the truth, it goes against the philosophy of installation. They rely on the presence of the right version of JRE. Some installations also package the native versions of JRE along with the installation package. This is quite an overhead. Other products use their custom homegrown installers. As most of the setup developers and administrators know, homegrown installers are a disaster waiting to happen. A good installation technology should do the following.

Installation - This is the easy part.
Rollback in case of failure
Support for dependency and conflict resolution.
Support for transactional install/uninstall operations. This is very important for the integrity of the system.
Support for upgrades.
Heal installed applications.

Creating a custom installation technology with all these features is difficult if not impossible. Also it is an investment that is not required when you can reuse the wheel. Each of the operating systems or the platform targeted have a native installation technology which support all of the above points in some way or the other. Most of these cross platform applications are deployed on Windows, Linux or Solaris. Windows has the Windows Installer a.k.a MSI package format to keep an inventory of software that is installed in an extremely fine granular level. Linux has RPM and DEB formats which deal with packages at a more coarse scale than Windows Installer but have excellent dependency and conflict resolution. Solaris has its native PKG format to install packages.

So instead of a cross-platform installation program, we could have a cross platform tool to create multiple installation packages for multiple platforms. One of the biggest problems working in this direction would be that the setup engineer should have a knowledge of all the platforms that he is packing the application for. This is usually not a easy job. For instance, I could handle Windows MSI and Linux's RPM & DEB files but would be a completely lost on a Solaris platform.

The biggest challenge would also be the fact that the customization scripts and custom actions would need to be coded by the setup engineer for each platform or package format. But this is a small price to pay for the amount of flexibility that can be achieved. The software deployment becomes a breeze and would significantly lower the (holy grail of three letter acronyms) TCO.

Anders Helsberg + Whiteboard + C# = really cool

Vagmi Mudumbai — Wed, 27 Apr 2005 14:36:00 GMT

While I was browsing the MSDN, I came across this very interesting video which features Anders Helsberg and his skills with the whiteboard. I really wish that he teach me programming. A must see for every C# programmer.

Writing managed code custom actions

Vagmi Mudumbai — Tue, 29 Mar 2005 01:55:00 GMT

With the new .NET framework API providing such wonderful functionality, everybody is programming in C# or VB.NET except setup developers. There was quite some commotion in the WIX users list on writing managed custom actions. It was finally decided that it is generally harmful to write managed code custom actions as they would depend on .NET Framework and having dependencies for setup is plain bad design. But IMHO, we can use managed code custom actions if we are 100% sure that it would be present on the target platform or if it is a prerequisite and a part of your LaunchConditions.

Writing managed code custom actions can be tricky. The easiest way is to implement the System.Configuration.Install.Installer class. This Installer class has four methods which can be executed during the MSI thread. The Installer class was only meant for developers to do some dirty stuff. But it is far from perfect to be used as a part of the MSI thread. Some of my strong reasons for not using the Installer class are as below.

InstallUtilLib.DLL is not completely silent. It still pops up an ugly message box with it fails.
The methods do not have access to the MSI thread. So we would not be able to use properties or write to the MSI log file.

We are then left with only one choice. Writing DLLs with C++ which export functions which use managed code. One of the several places that this could be used is handling XML configuration data. Windows Installer supports writing properties to both INI and Registry files but we do not have such facility for XML files. I guess this should be on the TODO list for the next version of Windows Installer <smile/>. But until then we would have to do it ourselves. There is a neat task in NAnt named <xmlpoke>. This custom action is programmed in pretty much the same way without namespace support. Given the path to the XML file and the XPath expression, we would be able to replace the value with our own using the installer. And it should also support properties.

I start off by creating a class library project in VS.NET 2003 and start editing the ProjectName.CPP file. Note that you would have to create a Module Definition file to support exporting the DLL functions that you write as symbols. Now for the fun stuff.

Lets first write functions that can write to the log file.

UINT WriteToLog(MSIHANDLE hMSI, CHAR *strMessage)
{

MSIHANDLE hrec=MsiCreateRecord(1);
MsiRecordSetString(hrec,0, strMessage);
MsiProcessMessage(hMSI,INSTALLMESSAGE_INFO,hrec);
MsiCloseHandle(hrec);
return ERROR_SUCCESS;

}

The above function is a pretty simple function that just writes any string of characters to the log file. While this function is good enough, we will write another function that accepts the managed String type.

void WriteToLog1(MSIHANDLE hMSI, String* strMessage)
{

CHAR* strMessageArr={0};
strMessageArr=(char*)(void*)Marshal::StringToHGlobalAnsi(strMessage);
WriteToLog(hMSI,strMessageArr);

}

This functions accepts the managed String* and Marshals it to a character array and then writes it to the log file. Now that we have the logging functions ready, lets concentrate on the actual juice. The idea here is that we use a deferred custom action to poke values into the XML file. The custom action expects data in a '|' delimited list in the following format.

[XMLFILEPATH]|[XPATHEXPRESSION]|[NEWVALUE]

A immediate type 51 custom action sets the CustomActionData property in the above format.

extern "C" __declspec(dllexport) UINT XmlPoke(MSIHANDLE hMSI)
{

TCHAR propval[261]={0};
DWORD len=261;
MsiGetProperty(hMSI,TEXT("CustomActionData"),propval,&len);
WriteToLog(hMSI,propval);
String *str=new String(propval);
String *delims="|";
Char delim[]=delims->ToCharArray();
String* split[]=str->Split(delim);
FileInfo* fi=new FileInfo(split[0]);
if(fi->Exists){

try{

XmlDocument* xdoc=new XmlDocument();
xdoc->Load(split[0]);
XmlNodeList * nl=xdoc->SelectNodes(split[1]);
IEnumerator * nodesenum=nl->GetEnumerator();
WriteToLog1(hMSI,split[1]);
while(nodesenum->MoveNext())
{

XmlNode* xNode=__try_cast<XmlNode*>(nodesenum->Current);
String* oldValue1=xNode->InnerXml;
WriteToLog1(hMSI,String::Concat(oldValue1, new String(" was the old value")));
xNode->InnerXml=split[2];
String* NewValue="New Value is ";
WriteToLog1(hMSI,String::Concat(NewValue,split[2]));

}

xdoc->Save(split[0]);

}

catch(Exception* exc){

WriteToLog1(hMSI,exc->Message);

}

}

else

{

WriteToLog(hMSI,"Error: The file does not exist");

}

return ERROR_SUCCESS;

}

The above function reads the CustomActionData property and splits it into an array. I assume that setup developers are good and they always give good data. So I have only gone for rudimentary error handling. The custom action never fails unless the error is catastrophic and is not handled by the try-catch block. It tries find if the XML file exists and if it does, it loads the XML file. I then use the SelectNodes() function to select the list of nodes as per the XPath expression. And finally I set the InnerXml value of the function to the new value. I also log these values as and when required. Thats it... as simple as it seems.

Dont forget to have the following header files and namespaces on top of the file. You would also have to include reference to System.Xml.Dll.

#include "stdafx.h"
#include "windows.h" //Required by MSI.h
#include "XmlTasksManaged.h"
//The MSI Stuff
#include "Msi.h"
#include "MsiQuery.h"

//.NET Stuff goes here
using namespace System;
using namespace System::IO; //For the FileInfo Object
using namespace System::Collections; //For the IEnumerator
using namespace System::Xml; //The XML Stuff
using namespace System::Runtime::InteropServices; //For the Marshal object

As mentioned earlier the Namespace support is still not included. It is not a very difficult functionality to add. We can pass it to the CustomActionData property. We can then cut and slice the CustomActionData property as we please. <smile/>

Unit Testing MSI Packages... Maybe

Vagmi Mudumbai — Thu, 17 Mar 2005 13:54:00 GMT

I was recently reading the article by Marting Fowler on Continuous Integration. Our team has set process to have daily builds of the MSI package along with the binary builds. It would have been an easier task with the WIX Toolset but most of our code still is in InstallShield DevStudio. We have used the InstallShield Standalone Build environment to build our MSI package as a batch job. We have various build machines to build different parts of the software and the Standalone Build environment helps us achieve this process without bleeding for separate licenses. As Marting Fowler had suggested, a successful build is just not a build without errors, it should be a workable output which clears some basic tests. I wondered if we could implement the Test First approach suggested by XP as a part of the MSI Build Process. The test first approach requires you to write self testing code which does some basic smoke tests. These are called unit tests or Build Verification Tests. I am sure that most developers include such code in their applications but I have not heard of such tests for Installation programs especially MSI packages.

I had blogged earlier on how the WIX Toolset can be used for distributed development. It is really cool that the WIX Toolset can avoid building merge modules, when they are not really required. But as we currently do not have much choice regarding this, we do build different projects as merge modules and later integrated them into the main project. Thus, technically our unit tests are integration tests.

A successful installation is not just an installation without errors. A successful installation should install the desired set of features and components. So, their states should be checked. This would involve parsing the verbose log file generated. The .NET Regular Expressions helps very well here <grin/>. We can check for the action return values and the feature and component installation states from the log file. Additionally, after installation, the files and registry entries on the target machine can be compared to the files and registry entries in the MSI package for the components that are marked installed. This can be done by comparing an uncompressed build or an administrative image with the files on the target machine. These criteria have a greater impact during upgrade than the first time installation.

I am just toying around with this idea and not really sure if it would be feasible for complex upgrade scenarios. I will post followups if I get anywhere with this. <smile/>

ATL RGS Woes

Vagmi Mudumbai — Thu, 10 Mar 2005 01:25:00 GMT

I had been off the blogging world for quite some time now. I was so swamped with work that I really could not collect my mind to write anything worthwhile. I recently bumped into a really sticky issue (and continue to remain stuck with that). We have a VSIP enabled COM DLL that uses a lot of ATL RGS code. Some of these values in this RGS file were dependent on some environment variables. Also the path to the satillite DLL and some project items were authored in the RGS file. I tried harvesting registry entries out of the RGS file but there were far too many parameters to handle. And hence, I had to do the unthinkable ---- SELF REGISTRATION --- . But despite our attempts, surprise..surprise..But that did not work either.

I then remembered that the environment variables were also a part of my package. And ofcourse, the environment variables will not be effected until the installation finishes and hence the registration failed. Developers did not have the time to fix it so we had to do a really nasty workaround. We needed to have chained setup; one to setup the environment variables and other to actually install the package. This is okay at the current stage as the package is used by our internal customers who are all developers and have predictable machine configurations. However, this will be a catastrophe on an actual production environment.

I had posted in the MS Visual Studio extensibility newsgroups about other alternatives but was informed by Bob Arson that RGS file was the only way to go. However, some of Microsoft's VSIP deployment presentations demostrated authoring these registry entries using the registry view in the Visual Studio Installer project. A little bit of googling also revealed that we could use VsipRegPkg.exe to extract these registry entries for managed plugins. But ours is an unmanaged plugin :-(. Our developers claim that they have done the right thing as per Microsoft's recommendation and that they would not require the environment variables after they acquire the PLK. But even then, there is the dependency on the satillite DLL to be present in a particular location. I wish that the other Microsoft teams run their suggestions via the deployment groups. This would greatly benifit the overall quality of the product. I guess our developers need to watch MSDN TV. They almost say "Use Managed Code" on every show.

Breeds of Setup Engineers

Vagmi Mudumbai — Thu, 06 Jan 2005 18:12:00 GMT

A little earlier, Chris had put this comment on my blog about Using The WIX Toolset for Distributed Development.

Firstly, I believe that Chris and I belong to different breeds of setup engineers. Chris uses tools like AdminStudio, DevStudio and Repackager to repackage setups and distribute them via SMS, Active Directory or other Change and Configuration management applications. This breed is often known as a packaging engineer. They have to often deal with hideous packages which move all sort of cryptic resources and perform changes on the system. Oh.. dont you hate repacking a setup while on a tight deadline when a custom configuration is performed on the target machine by the setup "script" which depends on the machine state/user input/configuration/location and data being moved?

I belong to a breed of setup engineers who are fortunate enough to design such hideous logic. We don't have to deal with black boxes of setup and try to figure out what is being installed. But we have a challenge of a different sort. We need to ensure that the setup runs without a problem on all configurations supported by the application itself. We are called Setup Developers. The setup developers have to take care of several criteria like the media size, privileges, future upgrades and design a fool proof system that is probably going to get upgraded and maintained for 3-4 years. And believe me, we do not have such a busy release schedule as InstallShield's. By the time we release a support package (which is usually a big patch), we forget what we put in the previous package. Hence we stay away from configuration specific tasks as far as possible. Maintaining such application/configuration data during the entire lifecycle of the product is quite a challenge.

Setup Developers are often found in packs. :-) Usually, more than one developer works on a setup package. My blog entry about using the WIX toolset for Distributed development is a real life scenario. So our efforts are that we create setups which can easily be deployed over a diaspora of machine configurations. If the setup fails then customer does not get a chance to find out the amazing bugs which the application developers put in to the product.

Configuration Data is such a Pain

Vagmi Mudumbai — Wed, 29 Dec 2004 23:16:00 GMT

I have finally got my computer to my new house in Bangalore. I dont have access to the internet, so I will essentially by offline. I am yet to install SharpDevelop and .NET Framework SDK v1.1. Hopefully, I should be able to get it done today. I had some time today morning but I was too busy playing NFS Hot Pursuit 2 and Quake 3. (No real souls to frag. Only bots. )

Currently one of our installation packages does something that it should not be doing. It is handling configuration data for several applications. Clearly this created several issues during upgrades. Now, for the next schedule major release, we (the packaging team) decided that we would let the applications handle the configuration information. This was duly presented as a proposal to other "application development" teams in a meeting. The people were a little hot under the collar during the meeting. Other development groups suspected that we were doing this to brush off our responsibility. I am sure that other setup developers would also have faced such similar predicament.

I was recently browsing through the InstallShield Community forums and found that mosth setup developers were invariably including configuration data as a part of their installation. I even saw one post where a setup developer had used a VBScript custom action with the FileSystemObject and tried to replace certain place holder texts in the installed configuration file. I am sure that many of us, who have burnt their fingers with VBScript and FSO, would agree with me. Logically speaking, there is nothing wrong with this approach. But we have to realize that we are introducing a element that can go wrong. Having deferred VBScript custom actions with FSO is a lot of fun while debugging.

My recommended approach would be to have a small configurator utility, which would accept command line parameters and perform the same tasks. This serves two purposes.

The configuration data is effectively taken out of the MSI. You can avoid slicing and dicing the CustomActionData property to get the desired values. There is one element less in the project that can go wrong.
The user need not run the installation, if he has specified incorrect configuration. He has the option of configuring the software after installing it. While installing on locked down environments, the setup author has to ensure that the permissions table is authored such that the current user has no problems writing on the the specified resources. This should'nt be a problem for registry keys or folders as their permission tokens can always be altered by the installer.

Finally, Riko is back. It seems that he is going to submit his tallow source to the community. I can't wait to get my hands on it.

My 2 cents

Vagmi Mudumbai — Thu, 23 Dec 2004 22:18:00 GMT

If people were following the the WIX-Users Mailing list, they would find that it has been a little hot recently. The first instance (as far as my knowledge goes) was with Michael creating a validation tool for the WIX Toolset called 'smoke'. I personally found it very useful as it could easily be included as a part of the build process. The output is in a structured XML format that makes it easier for us to crunch and build reports. But, some of the members in the group did not share the same level of enthusiasm. Michael seemed to be put off by that and included a disturbing update to his blog entry. Rob immediately acknowledged it and posted a blog entry about being thick skinned.

Recently, history repeated with Marc Bogers starting to ask about the ICE 33 warnings as Tallow populates the Registry table instead of the Registry Tables group. Firstly, Tallow is only a helper utility and is anything but perfect. It is useful but does not exactly produce usable WIX code. Heck, it doesn't even write to a file. Rob had said that ICE33 warnings can be ignored as it created some "strange behavior". More precisely, the resiliency associated with advertisement would prompt the end user for the media when Windows Installer detects some inconsistencies with the CLSID registration. Michael dismissed the argument that it arose from sloppy authoring and had nothing to do with MSI itself. Follow Michael's blog entry on COM registration for more information.

As we all know, Rob does not agree on a lot of things like inserting GUIDs while generating output with Tallow, using the Registry Tables Group and so on. I am sure that he has good reasons behind them. I am also sure that many people (including me), have modified Tallow to suit their requirements. Some people are using Riko's Tallow that does write to the Regsitry table's group using the Class and TypeLib elements. (On a totally different note, Riko, when are you going to release that code of yours?) I have a version of Tallow that creates components with GUIDs and names them appropriately. I am pretty sure that I am aware of the consequences and I have written code to persist the component information for future reference.

This freedom is the inherent beauty of open-source software. People can use their own ingenious imagination to tailor the tool to best suit their requirements. Of course, some of our opinions differ and we need a healthy debate to analyze these opinions and make the best of it. We should refrain from flaming and should try to embrace new ideas. Rob may be the BDFL for the WIX Toolset but I am sure that he would like the WIX Toolset to be successful and that helps developers create better installers without breaking much sweat. I tried hard not to write about this topic but after reading Michael's blog and comments with certain conspiracy theories, I wanted to voice my opinion too. I am sorry if I have hurt anybody's feelings.