Consultants (like me) tend to exist in very different worlds than enterprise architects and developers. We get exposure across a broad range of companies, flitting from site to site, solving problems, spewing wisdom, and moving on. That’s a great position to be in as you get to see a lot of different business problems and solutions. That’s also sometimes a not-so-good position as you see people struggling, often with the same problems, and coming up with solutions that can vary wildly in both approach and effectiveness.
In BizTalk-land, nowhere is this as evident as around deployment. Why, I don’t know, but many people seem to struggle with it. Developers will focus on making the solution work, and then suddenly realize they need to deploy, and wonder how they will go about doing that. This made more complex as BizTalk solutions can, and often do, span a wide range of technologies.
The BizTalk product team has come up with a great deployment story. This has not always been the case, but it certainly is now. We have all the bits and pieces we need to be able to implement highly-repeatable deployment packages. However, many developers seem to either not realize that, or they’re not in a position (due to budgets and tight deadlines) to take the time to “do it right”.
I would argue that not taking the time to implement a 100% repeatable build process is something that you CAN’T AFFORD to skip. The ultimate cumulative costs, in ghost-chasing troubleshooting, production outages, and lost productivity can far outweigh the up-front costs of implementing a highly-repeatable, reliable and effective build process. In some shops it may be OK to hand an IT pro a deployment script that has 15 manual steps, however, I’d rather not rely on a human to ensure that every step happens in the prescribed sequence. What if you miss step 12?
I know a lot of others (such as Scott Colestock) have done great focused work in the area of deployments. I embarked on this path somewhat by accident. I was on a client project where we were migrating MSIs, BizTalk’s preferred unit of deployment, between environments. I was doing this by exporting an MSI from the admin tool. One day, I forgot to update a binding file (a resource included with the MSI), and proceeded to chase ghosts for a little while when my import was nattering about ports I knew no longer existed. Frustrated, I promised myself this would never happen to me again, and I wrote a little batch file that would automatically update the binding resources for me, I wouldn’t have to remember, I’d just run an “UpdateBizTalkResources.cmd” script, and magic would happen.
That’s how it began, and it just kind of grew organically from there. This blog post is not about how it ends, but I want to share where I am at now. Ever project adds to this, and it has been an iterative process spanning several projects over the past couple of years.
So where am I at? At a client site last year I was able to take a somewhat simple application, and generate an MSI that reduces the deployment time of that robotics-control BizTalk solution to just a few minutes. Everything is scripted, everything is repeatable. At another client site last year, I took a solution that spans perhaps 20 BizTalk projects, and reduced creation of the MSIs (including compilation, etc) to running a series of scripts. Elapsed time to re-gen the 8 MSIs is under 15 minutes, deployment to a green-field pristine BizTalk environment, also under 15 minutes, for all 8 MSIs (which generate 5 separate BizTalk applications). The IT guys on that particular project loved me, as I took their deployments from being a two day "hope this works" process to a 15 minute highly-repeatable process with only a handful of steps.
To those fortunate few among you that have the luxury of being able to spend the time to implement highly-repeatable, highly-scripted, as-touch-free-as-possible build processes, this may seem pretty routine. However, a lot of BizTalk shops don’t have this degree of process implemented.
For the rest of you, I am posting my current script (with some client info removed, removed items are marked by "[" and "]"). Most of it should be self-explanatory, but at a high level, it:
- Creates a log file. All steps that occur, and their results, are recorded in the log
- Compiles the SLN file by calling DEVENV
- Removes the existing BizTalk app and uninstalls it (if it exists) to ensure a clean “not tweaked by a developer” build
- Creates a BizTalk application
- Adds the resources (including binding files for each environment)
- Exports an MSI
In some cases, you’ll need to do more when the application installs, which you can handle with pore and post-processing scripts. For example, in one case, I needed to move DLLs created by the SAP adapter into a specific folder. Just to make it challenging, the target folder changed with environments. Fortunately, I could detect the environment by querying an environment variable, and move the file accordingly. All of that was transparent to the IT Pro who just ran the MSI. In another case, all solution DLLs needed to be put into specific folder locations. In yet another case, while working on the ESB Guidance, Marty, Tom and I wrote scripts that automatically did everything from creating user accounts and app pools through to creating virtual roots and assigning app pools. Post-processing scripts are a powerful tool, you can do a LOT there.
In practice, I develop this script in my development VM. When the time comes to deploy to the integration environment (pre-QA/UAT), I am actually doing my first pass at testing the build/deploy process. Usually, the REAL MSI is created when the script is run on a "build server". That MSI moves into QA. If everything succeeds and is accepted in QA, then that same MSI (don't re-generate!) moves into production.
I hope you’ll find the techniques used below help improve the quality and repeatability of your builds, and help to simplify your build and deployment process. As always, I'd welcome any comments/thoughts you all may have on how this can be further improved.
---------------------------
@echo off
mode con cols=100 lines=120
CLS
SETLOCAL
SET APP_NAME=[name]
SET BIZTALK_APP_NAME=[app name]
SET BinPath=source code\bin
SET BasePath=[path]\Source Code
SET LOG_LOCATION=%CD%\LogFile.txt
REM Uncomment the following line to echo everything out to the screen
REM set LOG_LOCATION=con
SET Database="BizTalkMgmtDb"
SET Server="."
echo ##################################################################################### >> %LOG_LOCATION%
echo %DATE% %TIME% Running %APP_NAME% build script >> %LOG_LOCATION%
echo ##################################################################################### >> %LOG_LOCATION%
echo Setting up...
echo ############# >> %LOG_LOCATION%
echo Setting up... >> %LOG_LOCATION%
echo ############# >> %LOG_LOCATION%
CALL vsvars32.bat
echo Building....
echo ############# >> %LOG_LOCATION%
echo Building .... >> %LOG_LOCATION%
echo ############# >> %LOG_LOCATION%
cd %BasePath%
echo. >> %LOG_LOCATION%
echo ********************************************************************************************** >> %LOG_LOCATION%
echo *** Working in directory: %CD% *** >> %LOG_LOCATION%
echo ********************************************************************************************** >> %LOG_LOCATION%
echo. >> %LOG_LOCATION%
DEVENV ..\..\%APP_NAME%.BizTalk.sln /build Deployment >> %LOG_LOCATION%
IF %ERRORLEVEL% NEQ 0 (
SET msg=Could not build Visual Studio solution
GOTO ErrorRoutine)
rem To ensure we have a clean environment free of any legacy or manually added resources, try to remove and uninstall the app
echo Cleaning environment....
echo ######################## >> %LOG_LOCATION%
echo Cleaning environment.... >> %LOG_LOCATION%
echo ######################## >> %LOG_LOCATION%
BTSTask UninstallApp -ApplicationName:%BIZTALK_APP_NAME% >> %LOG_LOCATION%
BTSTask RemoveApp -ApplicationName:%BIZTALK_APP_NAME% >> %LOG_LOCATION%
BTSTask AddApp -ApplicationName:%BIZTALK_APP_NAME% >> %LOG_LOCATION%
IF %ERRORLEVEL% NEQ 0 (
echo ####################################################################### >> %LOG_LOCATION%
echo #### Warning: BizTalk application %BIZTALK_APP_NAME% already existed. >> %LOG_LOCATION%
echo #### Application is being 1-uninstalled and 2-removed >> %LOG_LOCATION%
echo ####################################################################### >> %LOG_LOCATION%
)
echo *** Adding assemblies generated by SAP adapter to %APP_NAME% resources. The post-processing script will move it to the right folder (destination varies by environment)
BTSTask AddResource /ApplicationName:%BIZTALK_APP_NAME% /Type:System.BizTalk:BizTalkAssembly /Source:"C:\Program Files\Microsoft BizTalk Adapter v2.0 for mySAP Business Suite\Bin\Z_MATERIAL_SPECS.dll" /Destination:"%%BTAD_InstallDir%%\Z_MATERIAL_SPECS.dll" /Server:%Server% /Database:%Database% /Overwrite >> %LOG_LOCATION%
IF %ERRORLEVEL% NEQ 0 (
SET msg=Could not add SAP DLL to BizTalk application
GOTO ErrorRoutine)
echo Adding binding files...
echo ####################### >> %LOG_LOCATION%
echo Adding binding files... >> %LOG_LOCATION%
echo ####################### >> %LOG_LOCATION%
BTSTask AddResource /ApplicationName:"%BIZTALK_APP_NAME%" /Type:System.BizTalk:BizTalkBinding /Server:%Server% /Database:%Database% /Overwrite /Source:"..\..\build\Bindings\%BIZTALK_APP_NAME%_Bindings_DEV.xml" /Property:TargetEnvironment="Development" >> %LOG_LOCATION%
IF %ERRORLEVEL% NEQ 0 (
echo #################################################################### >> %LOG_LOCATION%
echo #### Warning: could not find DEV binding file >> %LOG_LOCATION%
echo #################################################################### >> %LOG_LOCATION%
)
BTSTask AddResource /ApplicationName:"%BIZTALK_APP_NAME%" /Type:System.BizTalk:BizTalkBinding /Server:%Server% /Database:%Database% /Overwrite /Source:"..\..\build\Bindings\%BIZTALK_APP_NAME%_Bindings_QA.xml" /Property:TargetEnvironment="Q.A." >> %LOG_LOCATION%
IF %ERRORLEVEL% NEQ 0 (
echo #################################################################### >> %LOG_LOCATION%
echo #### Warning: could not find QA binding file >> %LOG_LOCATION%
echo #################################################################### >> %LOG_LOCATION%
)
BTSTask AddResource /ApplicationName:"%BIZTALK_APP_NAME%" /Type:System.BizTalk:BizTalkBinding /Server:%Server% /Database:%Database% /Overwrite /Source:"..\..\build\Bindings\%BIZTALK_APP_NAME%_Bindings_PRD.xml" /Property:TargetEnvironment="Production" >> %LOG_LOCATION%
IF %ERRORLEVEL% NEQ 0 (
echo #################################################################### >> %LOG_LOCATION%
echo #### Warning: could not find PRD binding file >> %LOG_LOCATION%
echo #################################################################### >> %LOG_LOCATION%
)
echo Creating MSI...
echo ############### >> %LOG_LOCATION%
echo Creating MSI... >> %LOG_LOCATION%
echo ############### >> %LOG_LOCATION%
BTSTask ExportApp -ApplicationName:%BIZTALK_APP_NAME% -Package:..\..\build\%BIZTALK_APP_NAME%.msi >> %LOG_LOCATION%
IF %ERRORLEVEL% NEQ 0 (
SET msg=Could not export MSI
GOTO ErrorRoutine)
GOTO :End
:ErrorRoutine
echo #################################################################### >> %LOG_LOCATION%
echo #################################################################### >> %LOG_LOCATION%
echo Error: %msg% >> %LOG_LOCATION%
echo #################################################################### >> %LOG_LOCATION%
echo #################################################################### >> %LOG_LOCATION%
GOTO :End
:End
echo. >> %LOG_LOCATION%
echo. >> %LOG_LOCATION%
echo ##################################################################################### >> %LOG_LOCATION%
echo --> Completed: %DATE% %TIME% Running %APP_NAME% build script >> %LOG_LOCATION%
echo ##################################################################################### >> %LOG_LOCATION%