Backing up SQL Azure

June 21, 2011 · 1 min read

That's it!!! After many days and nights... and an amazing set of challenges, I just released the Enzo Backup for SQL Azure BETA product . Clearly, that was one of the most challenging projects I have done so far.

Why???

Because to create a highly redundant system, expecting failures at all times for an operation that could take anywhere from a couple of minutes to a couple of hours, and still making sure that the operation completes at some point was remarkably challenging. Some routines have more error trapping that actual code...!(https://geekswithblogs.net/Providers/BlogEntryEditor/FCKeditor/editor/images/smiley/msn/regular_smile.gif)

Here are a few things I had to take into account:

  • Exponential Backoff (explained in another post)
  • Dual dynamic determination of number of rows to backup 
  • Dynamic reduction of batch rows used to restore the data
  • Implementation of a flexible BULK Insert API that the tool could use
  • Implementation of a custom Storage REST API to handle automatic retries
  • Automatic data chunking based on blob sizes
  • Compression of data
  • Implementation of the Task Parallel Library at multiple levels including deserialization of Azure Table rows and backup/restore operations
  • Full or Partial Restore operations
  • Implementation of a Ghost class to serialize/deserialize data tables

And that's just a partial list... I will explain what some of those mean in future blob posts. A lot of the complexities had to do with implementing a form of retry logic, depending on the resource and the operation.

This article is part of the GWB Archives. Original Author: Herve Roggero

New on Geeks with Blogs

  • We Won The One Award I Actually Care About

    Full Scale made the Inc. 5000 for the fifth year straight, the 12th listing across my three companies. Here is why the one award you cannot buy is worth stopping for.

  • Your Customers Build the Features Now

    I let a tool I liked sit dead for a year rather than build the features I wanted. An MCP server meant I never had to, and your customers can do the same to your product.

  • Get the Size of a Directory in Linux the Easy Way

    du -sh for the quick answer, ncdu for the cleanup, df for the disk itself: every command for checking directory size in Linux, plus why du and df never agree.

  • Vim Search and Replace: The Ultimate Guide

    One :%s command replaces every match in a file before a find dialog would even open. The Vim substitute patterns worth the muscle memory: flags, ranges, capture groups, and multi-file edits.