DropkicK–Deploy Fluently

DropkicK (DK) has been in development for over two years and has been used for production deployments for over a year. Dru Sellers originally posted about DK back in 2009. While DK isn’t yet as super easy to grok as some of the other ChuckNorrisFramework tools and offers little in the idea of conventions, it is still a stellar framework to use for deployments.

DK works well in environments where you know all of the environments you will deploy to ahead of time (although not required due to the ability to pull in JSON settings files and servermaps). It is not for every environment, as DK will need to be able to get to the remote location through UNC (if deployed from a local server everytime it won’t be an issue) except for the database. DK is continually improving, so expect a transition into adding FTP type deployments as well.

I am going to stay somewhat introductory, so you won’t see this post get too detailed into exactly how you can use DK for deployments. That would be best covered by reading the wiki and looking at examples or a series of articles.

Concepts of Kicking Your Code Out with DropkicK

Deployment Step – The simplest concept of execution of deployment. This is a step that is involved with getting something set up during a deployment. This could be copying files or setting a folder permission.

Deployment Task – This is a collection of one or more steps to do in making something happen during the deployment. Say a task is to copy some files. A step in that task might be to clean/clear folders. Another step is to remove read only attributes. The last step in that is to actually copy files/folders. This is nearly synonymous with the concept of deployment steps and often referred to that way, even by the maintainers of DK.

Deployment Role – A role is a collection of tasks that as an atomic unit have set up a particular area of a deployment. Like a database. Or a Web site. A role contains one or more deployment tasks.

Deployment Plan – This is a collection of all roles for making a deployment happen. This is what you write when you sit down to write a dropkick deployment for your code.

Deployment Settings – These are settings you can draw from in any deployment step. A core concept to DK is the idea of environments and is baked into all settings.

Deployment JSON Settings – This is the equivalent of the deployment settings, with the actual values that you want the deployment settings to get at run time. This is separate so that you can make changes in case you need to make changes prior to deployment.

Deployment Server – A deployment role is targeted against one or more servers.

Deployment ServerMaps – This is the physical server or servers that you want to target Deployment Roles to for a particular environment. Each role you want to deploy will need at least one physical location.

Remote Execution – When certain tasks must be run against the server they are targeting, DK will copy over an executable to a known location on that machine, run it through WMI on that particular machine, wait for it to finish, and then bring the execution log back to the main logs. This means you do not need a service installed on the remote machine for installation.

Deployment Logs – DK has a few logs that it puts together during the deployment. The one you see in the console is a summary of what is happening. There is a run log that contains details of everything that is happening. There is also a db log, a security log, and a file change log. These logs can be passed to each party that cares about them after a deployment for auditing sake.

NuGet Install

If you want to get a quick start on seeing a good example of DK, just pull in the dropkick nuget package and it will bring some sample code.

Running DropkicK

DropkicK expects you to tell it where the deployment DLL file is, what environment it is deploying to, what roles it is deploying, and where the deployment settings files are located. It runs in trace mode by default, determining if one can actually execute the deployment plan (has permissions, servers exist, etc).

The syntax for running dropkick is:

dk.exe [command] /environment:ENVIRONMENT_NAME [/ARG_NAME:VALUE] [--SWITCH_NAME]

At a minimum you can run dropkick with dk.exe execute. This will deploy all roles to ‘LOCAL’ environment with ‘Deployment.dll’ (seated next to dk.exe) looking for ‘.\settings\LOCAL.servermaps’ and ‘.\settings\LOCAL.js’

dk.exe execute /deployment:..\deployments\somename.deployment.dll /environment:LOCAL /settings:..\settings /roles:Web,Host

The above should give you an idea of all of the options you can pass to DK for execution.

You can pass a silent switch to DK to allow for completely silent deployments. Although rough at the moment, there is a wiki article for deploying from TeamCity. That can be found here: https://github.com/chucknorris/dropkick/wiki/TeamCityIntegration

Enough Talk - Show Me the Code!

The below code shows an example deployment plan for executing a deployment to Db, Web, and Host roles. It leaves out the Virtual Directory setup, but that can be easily brought in from looking at an example (https://github.com/chucknorris/dropkick/blob/master/product/dropkick.tests/TestObjects/IisTestDeploy.cs).

using System.IO;
using System.Security.Cryptography.X509Certificates;
using dropkick.Configuration.Dsl;
using dropkick.Configuration.Dsl.Files;
using dropkick.Configuration.Dsl.Iis;
using dropkick.Configuration.Dsl.RoundhousE;
using dropkick.Configuration.Dsl.Security;
using dropkick.Configuration.Dsl.WinService;
using dropkick.Wmi;

namespace App.Deployment
{
public class TheDeployment : Deployment<TheDeployment, DeploymentSettings>
{
  public TheDeployment()
  {
      Define(settings =>
      {
          DeploymentStepsFor(Db,
                             s =>
                             {
                                 s.RoundhousE()
                                     .ForEnvironment(settings.Environment)
                                     .OnDatabase(settings.DbName)
                                     .WithScriptsFolder(settings.DbSqlFilesPath)
                                     .WithDatabaseRecoveryMode(settings.DbRecoveryMode)
                                     .WithRestorePath(settings.DbRestorePath)
                                     .WithRepositoryPath("https://github.com/chucknorris/roundhouse.git")
                                     .WithVersionFile("_BuildInfo.xml")
                                     .WithRoundhousEMode(settings.RoundhousEMode);
                             });

          DeploymentStepsFor(Web,
                             s =>
                             {
                                 s.CopyDirectory(@"..\_PublishedWebSites\WebName").To(@"{{WebsitePath}}").DeleteDestinationBeforeDeploying();

                                 s.CopyFile(@"..\environment.files\{{Environment}}\{{Environment}}.web.config").ToDirectory(@"{{WebsitePath}}").RenameTo(@"web.config");

                                 s.Security(securityOptions =>
                                 {
                                     securityOptions.ForPath(settings.WebsitePath, fileSecurityConfig => fileSecurityConfig.GrantRead(settings.WebUserName));
                                     securityOptions.ForPath(Path.Combine(settings.HostServicePath, "logs"), fs => fs.GrantReadWrite(settings.WebUserName));
                                     securityOptions.ForPath(@"~\C$\Windows\Microsoft.NET\Framework\v4.0.30319\Temporary ASP.NET Files", fs => fs.GrantReadWrite(settings.WebUserName));
                                     if (Directory.Exists(@"~\C$\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files"))
                                     {
                                         securityOptions.ForPath(@"~\C$\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files", fs => fs.GrantReadWrite(settings.WebUserName));
                                     }

                                     securityOptions.ForCertificate(settings.CertificateThumbprint, c =>
                                     {
                                         c.GrantReadPrivateKey()
                                             .To(settings.WebUserName)
                                             .InStoreLocation(StoreLocation.LocalMachine)
                                             .InStoreName(StoreName.My);
                                     });

                                 });
                             });

                             
          DeploymentStepsFor(Host,
                             s =>
                             {
                                 var serviceName = "ServiceName.{{Environment}}";
                                 s.WinService(serviceName).Stop();

                                 s.CopyDirectory(@"..\_PublishedApplications\ServiceName").To(@"{{HostServicePath}}").DeleteDestinationBeforeDeploying();

                                 s.CopyFile(@"..\environment.files\{{Environment}}\{{Environment}}.servicename.exe.config").ToDirectory(@"{{HostServicePath}}").RenameTo(@"servicename.exe.config");

                                 s.Security(o =>
                                 {
                                     o.ForCertificate(settings.CertificateThumbprint, c =>
                                     {
                                         c.GrantReadPrivateKey()
                                             .To(settings.ServiceUserName)
                                             .InStoreLocation(StoreLocation.LocalMachine)
                                             .InStoreName(StoreName.My);
                                     });
                                     o.LocalPolicy(lp =>
                                     {
                                         lp.LogOnAsService(settings.ServiceUserName);
                                         lp.LogOnAsBatch(settings.ServiceUserName);
                                     });

                                     o.ForPath(settings.HostServicePath, fs => fs.GrantRead(settings.ServiceUserName));
                                     o.ForPath(Path.Combine(settings.HostServicePath,"logs"), fs => fs.GrantReadWrite(settings.ServiceUserName));
                                     o.ForPath(settings.ServiceWorkDirectory, fs => fs.GrantReadWrite(settings.ServiceUserName));
                                     o.ForPath(settings.ServiceTriggerWatchDirectory, fs => fs.GrantReadWrite(settings.ServiceUserName));
                                     o.ForPath(settings.SecureWorkDirectory, fs => 
                                          { 
                                              fs.GrantReadWrite(settings.ServiceUserName);
                                              fs.RemoveInheritance();
                                              fs.Clear().Preserve(settings.ServiceUserName)
                                                  .RemoveAdministratorsGroup()
                                                  .RemoveUsersGroup();
                                          });
                                 });
                                 s.WinService(serviceName).Delete();
                                 s.WinService(serviceName).Create().WithCredentials(settings.ServiceUserName, settings.ServiceUserPassword).WithDisplayName("servicename({{Environment}})").WithServicePath(@"{{HostServicePath}}\servicename.exe").
                                     WithStartMode(settings.ServiceStartMode)
                                     .AddDependency("MSMQ");

                                 if (settings.ServiceStartMode != ServiceStartMode.Disabled && settings.ServiceStartMode != ServiceStartMode.Manual)
                                 {
                                     s.WinService(serviceName).Start();
                                 }
                             });
      });
  }

    //order is important
    public static Role Db { get; set; }
    public static Role Web { get; set; }
    public static Role Host { get; set; }
}
}

You might immediately see how this really sets up an environment. The biggest ideas in DropkicK are that you can specify a complete setup from nothing to having a machine completely set up.

RoundhousE–Intelligent Database Migrations And Versioning

“Because everyone wants to kick their database, but sometimes kicking your database is a good thing!”

Many would not argue that you should version your code, and few would argue against versioning your code in a way that can lead back to a specific point in source control history. However, most people don’t really think of doing the same thing with your database. That’s where RoundhousE (RH) comes in.

I have been working on RH for over two years now and people always wander what it is, why and what sets it apart from other migrators. We set out to make a smart tool for migrations that came somewhat close to Ruby’s ActiveRecord Migrations without going the code migrations route (yet). Hopefully this introduction will help you understand why it is different and whether it’s something that is in line with your needs.

What is RoundhousE?

RoundhousE (http://projectroundhouse.org) is a database migrator that uses plain old SQL Scripts to transition a database from one version to another. RoundhousE currently works with Oracle, SQL Server (2000/2005/2008/Express), Access, MySQL, and soon SQLite and PostgreSQL. It comes in the form of a tool, MSBuild, and an embeddable DLL. While someone is working on a GUI, there is no visual tool at the current time.

RoundhousE - Kick It!

What sets RoundhousE apart from other migrators?

It subscribes to the idea of convention over configuration, which means you can pass the migrator very few configuration options to get it to work (rh.exe /d dbname), but pass as many options as necessary to meet your conventions. Say you don’t like the tables or folder names that RH uses, you can override those to whatever you want.

RH versions the database how you want it versioned. You can supply it with a DLL path for it to pull the file version from. You can give it an XML file and XPath, or you can use the highest script number in the up folder. You can also just use a sequence based (non-global) form of passive versioning. https://github.com/chucknorris/roundhouse/wiki/Versioning

RH believes in low maintenance and keeping good clean history in your source control. This means that you don’t lump everything into one folder, you put your anytime scripts (views/functions/stored procedures/etc) into their own folders and track history as you go. RH is smart enough to only run these if they are new/different from the current existing scripts in the database.

RH has three modes of operation. Normal, DropCreate, and Restore. Notice none of those are Create like you may see in other migrators. If the intent in the end is to have a database ready to go, why would you want to have to make a step to specify that you want to create the database? RH is smart enough to realize that the database doesn’t exist and it creates it (unless you pass a switch explicitly telling it not to). Normal is just the migration as it is. DropCreate is used during development when you want to continually change the same scripts prior to production. Restore is used when you switch to maintenance mode and want to change the same maintenance script. https://github.com/chucknorris/roundhouse/wiki/RoundhousEModes

RH is environment aware, which means you can have environment specific scripts. If you have scripts or permissions scripts that are different for each environment you can give them a special name.  https://github.com/chucknorris/roundhouse/wiki/EnvironmentScripts

RH is an easy to start using on legacy databases. You just take your old DDL/DML scripts and move them into a special folder that RH will only evaluate/run when it is creating a database (say on a new developers machine). You can arrange existing scripts into RH default folders or point RH to the existing folder types. RH splits scripts with the GO batch terminator in them.

RH speeds up your development process. You can use RH with NHibernate to refresh your database without leaving Visual Studio! Entity Framework and FluentMigrator are planned for this feature as well. https://github.com/chucknorris/roundhouse/wiki/Roundhouserefreshdatabasefnh

RH runs on just the .NET framework. This means you don’t need SMO installed like some other migrators require.

While there are probably other features I haven’t mentioned, keep in mind that RH is not a code migrator (yet). If you are looking for a code migrator, there are quite a few good tools out there, including FluentMigrator and Mig#. Entity Framework Code Migrations is really starting to shape up as well (Seriously! Although EF only works for SQL Server).

How do I get RoundhousE?

There are several avenues to get RH. You can use NuGet, Chocolatey, Gems, plain old downloads (still considered official releases), or source (both in git and svn). https://github.com/chucknorris/roundhouse/wiki/Getroundhouse