BizTalk: Deployment Hell & BizTalk Deployment Framework

Is there something special in the BizTalk Server application deployment? Why is it so special?

BizTalk Deployment Hell

For the .NET applications the live is simple. There is an exe file and maybe several additional dll-s. Copy them and this is pretty much all deployment we need.

The BizTalk Server requires the dll-s placed in GAC and registered in the special Management database. Why this is required? Mostly because the BizTalk automatically hosts applications in the cluster and because of the reliability. BizTalk application maybe is not easy in deployment but it also not easy to break. For example, if the application A is related to the application B, BizTalk will prevent us from removing B accidentally.

This Why? question is a big theme and I will not cover it here.

Another factor is the BizTalk has many pieces that require a special treatment in deployment and run-time.

Yet another factor is the BizTalk application integrates the independent applications / systems, that means the application should take care of many configuration parameters, as endpoint addresses, user names and passwords and so on.

As a result the BizTalk application deployment is complex, deployment is error prone, slow, unreliable, requires a good amount of resources.

And look here, it is a savior, the BizTalk Deployment Framework.

The BizTalk Deployment Framework - the Champion

The BizTalk Deployment Framework (BTDF) is an essential tool in arsenal of the BizTalk Server developers and administrators. It solves many problems, it speeds up the development and deployment enormously.

It is a definitely a main face in the BizTalk Server Hall of Fame. 

BTDF was created by Scott Colestock and Thomas Abraham.

It is an open source project despite the fact, that BTDF is more powerful, reliable and thorough than the most of commercial BizTalk Server third-party tools. I think it is fair to donate several dollars to those incredible guys on the Codeplex. Just think abut days and months we save in our projects, in our private life.

BTDF is an integration tool and it is created by guys with pure integration mindset. It integrates the whole bunch of open-source products in one, beautiful BTDF. Below I copy the "Contributors and Acknowledgements" topic from the BTDF Help:

Thanks also to:

  • Tim Rayburn for contributing a bug fix and unit tests for the ElementTunnel tool
  • Giulio Van for contributing ideas and bug fixes.
  • The hundreds of active users across the world who have promoted the Deployment Framework, reported bugs, offered suggestions and taken the time to help other users! …”

And how exactly BTDF saves our life?

BTDF is installed and tuned up. Now it is time to deploy an application.

DeploymentMenu.Deploy

The deployment was successful and I’ve got the long output, what exactly was done in this deployment.

Here is a log. I've marked my comments in the deployment log with “+++”. The full redeployment lasted 2 minutes.

If I perform the same amount of tasks manually, I would spent at last  3-4 additional minutes fully concentrated on this long list of deployment tasks. The probability of missing some step would be quite high as the probability of errors. With BTDF I start deployment and free to do another tasks.

So my gain is 3+ minutes on each deployment. Risk of an error is zero, everything is automated.

There is one more psychological problem with manual deployment. It is complicated and it requires the full concentration. As a developer I am concentrated on the application logic and a manual deployment task breaks my concentration. Each time after deployment I have to concentrate on my application logic again. And here is where the development performance goes down to the hell.

There are other helpful BTDF commands:

DeploymentMenu.All

  • Restarts all host instances, only those are used in this application; restart IIS, if I deploy web-services, all with the "Bounce BizTalk" command.
  • Terminate all orchestration and messaging instances remained after tests.
  • Install the .NET component assemble into GAC.
  • Include the modified configuration parameters into a binding file.
  • Import this binding file.
  • Update SSO with modified configuration parameters.

All this with one click.

You do not have to use the slow BizTalk Administrative Console to do those things anymore.

Check my comments in the BTDF deployment log. Several automated tasks are like blessing! We do not create the drop folders anymore, do not assign permissions to those folders. BTDF makes this automatically.

We do not care about undeployment and deployment order, BTDF makes everything right.

We do not stop and start ports, orchestrations, applications, host instances, IIS. Everything is automated.

Look at the list of the top level parameters in BTDF:

DeploymentProperties

Remember, I told you the BizTalk application deployment is a little bit complicated? All those application components in this list require a little bit different treatment in deployment. In a simple application we do not use all those components, a typical application uses maybe 1/3 or 1/2 of them, but you have an idea.

How to tune up BTDF?

One thing I love in BTDF is the Help. It is the exemplary, ideal, flawless help.

If you never tried BTDF, there is a detailed description of all parts, all tasks. Moreover there is the most unusual part, the discussions of the BTDF principles and the BizTalk deployment processes. I got there more knowledge about the BizTalk Server deployment than in the Microsoft BizTalk Server Help - official documentation.

BTDF Help helps if you are a new user or if you use BTDF several years in row. Descriptions are clear, they are not stupid and arranged in clear hierarchy. BTDF Help is one of the best. You are never lost.

Of course there is a detailed tutorial in Help and the sample applications.

OK Now we have to start with tuning.

Typical BTDF workflow for setting up a deployment of a BizTalk application:

  1. Creating a BTDF project.
  2. Setting a deployment project.
  3. Creating an Excel table with configuration parameters
  4. Setting a binding file

All configuration parameters are managed inside an Excel table:

Forget about managing different binding files for each environment. Everything is inside one Excel table. BTDF will pass parameters from this table into binding file and other configuration stores.

Excel helps when we compound parameters from several sources. For example we keep all file port folders under one root. The folder structure below this root is the same in all environments, only the root itself is different. So there is a parameter "RootFolder" and we use it as a part of the full folder paths for all file port folders. For example, we have a "GLD_Samples_BTDFApp_Port1_File_Path" parameter which defined in the cell with formula: = "RootFolder" + ""GLD_Samples_Folder" + "BTDFApp_Folder" + "Order\\*.xml" (of course in the Excel cell it would be something like = C22 & C34 & C345 & "Order\\*.xml"). If we modify the RootFolder path, all related folder paths will be modified automatically.

OK We work hard setting up the application development. Now surprise, we have to create a new environment. The best part of BTDF configuration fiesta is there: all configuration parameters for ALL ENVIRONMENTS for an application are here just in one table. (If you are tenacious enough you could keep a single table for ALL applications, but you have to ask me how.)

Settings for different environments: In our Excel table we copy-past a column of one of the existed environment to a new column for a new environment. Then we modify the values in this new column. And again, this is an Excel table, and if we define single RootFolder variable for a new environment and voila, all file port paths for this environment are modified.

Now we have to pass those configuration parameters to the binding files, right?

We replace all values in the binding file which are different for different environments, like the Host names, NTGroupName, Addresses, transport parameters like connection strings, etc. We replace these values with the variables, like this:BindingFileParameters

Now we save this binding file with the PortBindingsMaster.xml name.

That is pretty much everything we need. Now execute the Deploy BizTalk Application command and application is deployed.

Deployment into a Production environment is different. It is limited in the installed tools and we have to do additional installation steps. BTDF creates an msi and command file. This msi includes all additional pieces we need to install. We don’t have manually add resources to the BizTalk application in the Administrative Console anymore.

Conclusion: BizTalk Deployment Framework is a mandatory tool in the BizTalk development. If you are a BizTalk developer you must use it, you must know it.

Print | posted on Sunday, February 2, 2014 7:48 PM

Feedback

# re: BizTalk: Deployment Hell & BizTalk Deployment Framework

left by Prabhdeep at 2/21/2014 12:54 AM Gravatar
Thanks Leonid,
I have been using BTDF, for past 6-7 months and needless to say, my development time has decreased 3 fold. Its not just 3 minutes you save during development (gac dlls etc) but the ease of BTDF. Just click Quick deploy and it replaces all dll in gac. And then bounce shortcut will re-start all host -instances mentioned in .btdfproj file.
Although all this helps during development, I would like to add that using combination of BTDF and PowerShell scripts,we have achieved one click undeployment/ Re-deployments and hence saving a lot of time/energy for Admins.

# re: BizTalk: Deployment Hell & BizTalk Deployment Framework

left by Leonid Ganeline at 2/21/2014 5:27 AM Gravatar
@Prabhdeep: thank you for feedback!
BizTalk Server PowerShell Provider (http://psbiztalk.codeplex.com/) is one more great tool for sure. In BizTalk 2013 it is a part of BizTalk, which shows how important it is. Especially it helps in complex deployment scenarios.

# re: BizTalk: Deployment Hell & BizTalk Deployment Framework

left by Tim Riley at 5/1/2014 6:41 AM Gravatar
I agree that BTDF is by and large a very useful product. The one thing I don't understand is why the developers chose to move away from BizTalk installation packages.

# re: BizTalk: Deployment Hell & BizTalk Deployment Framework

left by rvs krishna at 4/28/2015 6:28 AM Gravatar
I hardly know about biztalk as im working on it for only 3 months but i have some knowledge (building and keeping the .Dll file in GAC and deploying the whole app) but now i need to use this BTDF that i'm not familiar.can anyone mail me any BTDF related docs,white papers anything useful plzzz............
"And i already went through documentation provided and tried one sample and im getting unkown errors from that"
Post A Comment
Title:
Name:
Email:
Comment:
Verification: