At least once per week I'm asked about how to size a BizTalk Server implementation. While there are several published sources on performance metrics and sizing, the problem is that every BizTalk application has its own performance characteristics and as a result there is no formula or guideline that will provide developers with a reliable answer to their question.
But I've always been concerned that we were asking the wrong question, and performing our testing in a fashion that does not truly represent the maximum sustainable throughput of our BizTalk applications. When Wayne Clark published his blog article "Understanding BizTalk Server 2004 SP1 Throughput and Capacity" in April I finally realized what was wrong with our performance testing approach, and decided that I would write a testing tool to help our developers produce consistent, reliable capacity and throughput analysis on their implementations.
The result is the BizTalk Throughput and Capacity Tester, a standalone application that enables developers to produce a steady load of messages for consumption by a BizTalk solution and to monitor the proper system performance metrics during this testing. The application follows the guidelines outlined in Wayne's blog and is intended to answer the question "What is the maximum sustainable load that my BizTalk implementation can reliably process?" Before trying to run the application I highly recommend that you read Wayne's article to understand the metrics being analyzed and the performance testing approach used; anything short of a full understanding of the metrics and the approach could result in an invalid perspective of the results and might produce false performance expectations.
The application is nearing a Beta 1 release; I've issued two Release Candidates to a limited audience of developers within Avanade and fixed all reported bugs. I expect to post the Beta 1 release later this week and will make it available for download here at that time. But if you're like me you just have to see a screenshot to understand the application.
Here you see the main UI which displays a large graph of the relevant performance metrics of your implementation. The graph displays seconds across the Y axis (the 180 represents 3 minutes), 0 to 100 on the X(1) axis to repesent either 0 to 100 percentage or 0 to 100 in quantity, and 0 to 10,000 on the X(2) axis to represent 0 to 10,000 in quantity. In the middle of the chart the list of metrics you are currently monitoring is displayed. At the bottom is a tab control that allows the user to control various settings of the chart (such as refresh interval for the chart and the overall duration that the chart displays), the file production (including the number of files to produce per second, the output path for the files, and the path where processed files can be found and deleted), and to display details about the metric currently selected in the list above the tab control.
The following metrics can be monitored with the application:
- CPU Utilization
- Disk Utilization
- SQL Lock Timeouts
- SQL Lock Wait Time
- BizTalk Messages Received
- BizTalk Messages Processed
- BizTalk Spool Depth
All of the metrics are WMI performance counters, but this complexity is hidden from the user. You simply select the metric you wish to add to the list, pick the server to be monitored from the domain list, and the list of instances will be presented. Select an instance, and then set the line color and style for the chart.
This beta will be considered feature complete for release 1, but I'm interested in feedback on additional features you feel you would need to appropriately determine throughput and capacity testing for your applications.