BAM Archiving/BAM_DM_ observations

May 21, 2009 · 2 min read

From the BizTalk documentation: “The DTS package, BAM_DM_, performs the partitioning and archiving/purging. Each time this package runs, it truncates another partition and archives/drops all partitions that are outside the online window.”

Here’s my restatement of this:  The BAM_DM_ package (SSIS or DTS depending on your version of BizTalk and despite the fact that the docs refer to DTS only) performs two operations: building partition tables and archiving those partitions to the BAMArchive database when a partition’s creation date is past the online window.

If you open up a BAM_DM_ SSIS/DTS package and walk through it you’ll see that the second step calls the bam_Metadata_SpawnPartition stored procedure.  Toward the end of that sproc it inserts a new record into the bam_Metadata_Partitions table setting the CreationTime’s value but not the ArchivedTime value which means it will be NULL.

A later step in the package calls the bam_Metadata_BeginArchiving stored procedure.  Toward the end of this stored procedure it performs an SQL Update on the bam_Metadata_Partitions table where the ArchivedTime is null and the CreationTime is less than the online threshold (derived from the online window.)  If the Update succeeds any partitions that should be archived have their ArchivingInProgress value set to 1 (this is set so that if the package runs again, before it completes, the second run will essentially just exit without doing anything.)

Later steps (two of them) in the package perform the actual archiving; they examine the CreationTime and ArchivedTime to figure out what they should archive, using the same logic outlined above (for the . bam_Metadata_BeginArchiving sproc).

There are many more details but hopefully provides enough detail for the takeway.

Takeaway: your BAM_DM_ package almost certainly needs to run twice to actually archive data to the BAMArchive database because the time between when the partition is created during the second step of the package and when archiving will happen (later steps) isn’t going to be greater than your online window.  The lowest you can set your online windows is 1 minute.  Unless you have massive quantities of BAM data, and a slow SQL Server, it isn’t going to take more than a minute between when the bam_Metadata_SpawnPartition sproc runs (which sets the CreationTime) and the archiving steps execute.  So, the first time you run your package it will create the partitions; the second time it will archive those partitions, if they fall outside the online window.  This means you need to think carefully about how often you schedule your BAM_DM_ packages and how that is related to the online window.

For additional context reference the Business Activity Monitoring in Depth for Developers whitepaper here:  http://download.microsoft.com/download/D/D/A/DDA95D1F-14D4-49A2-B2C8-E244535E8502/BAM\_for\_Developers.docx

This article is part of the GWB Archives. Original Author: Andy Morrison

New on Geeks with Blogs

  • We Won The One Award I Actually Care About

    Full Scale made the Inc. 5000 for the fifth year straight, the 12th listing across my three companies. Here is why the one award you cannot buy is worth stopping for.

  • Your Customers Build the Features Now

    I let a tool I liked sit dead for a year rather than build the features I wanted. An MCP server meant I never had to, and your customers can do the same to your product.

  • Get the Size of a Directory in Linux the Easy Way

    du -sh for the quick answer, ncdu for the cleanup, df for the disk itself: every command for checking directory size in Linux, plus why du and df never agree.

  • Vim Search and Replace: The Ultimate Guide

    One :%s command replaces every match in a file before a find dialog would even open. The Vim substitute patterns worth the muscle memory: flags, ranges, capture groups, and multi-file edits.