Geeks With Blogs

News This is the *old* blog. The new one is at blog.sixeyed.com
Elton Stoneman
This is the *old* blog. The new one is at blog.sixeyed.com

This is the last one for the time being – how to get blob metrics for a Storage Account using the Azure Management Libraries, in the same way that I covered Cloud Services and Web Apps.

Storage is different from compute in terms of what data is available, and what it is you want to see. I still want overall status and how hard the account is working, but instead of scale I want to see how much is being stored and how much the account is being accessed:

image

The status block on the left tells me the name of the account, the level of redundancy and the current status. Redundancy ranges from Standard_LRS - meaning standard performance, locally-redundant within a single Azure data centre, to Premium_GRS – meaning premium performance, globally redundant within and across Azure data centres.

The metrics block tells me how many requests the account has served. I’m focusing on blob storage here, which is my immediate need, so this is telling me how many blob download and upload requests have been made. And the usage block shows me how much storage I’ve used, how many blobs I have stored in the account, and how many containers they’re spread across.

In daily use the requests metric is very handy, and when you’re reviewing an Azure subscription, it’s a great way to see where the data is concentrated across the Storage Accounts.

Storage Account Details

For the general account details, we can get everything using the StorageManagementClient, which (at version 3.0.0.0 of Microsoft.WindowsAzure.Management.Storage) uses the old credentials class and the CloudContext to create a client object:

var model = new BlobStorage();
using (var client = CloudContext.Clients.CreateStorageManagementClient(credentials))
{
    var respose = await client.StorageAccounts.GetAsync(_serviceName);
    var account = respose.StorageAccount;
    model.Name = account.Name;
    model.Redundancy = account.Properties.AccountType;
    model.Status = account.Properties.Status.ToString();
}

The StorageAccount class contains all the key details, like the name and URI, account type (which is the redundancy level). The StorageAccountProperties object contained in the StorageAccount has extra information, like the individual location URIs for geo-replicated accounts, the last failover time, and the status for the account and each region.

Blob Metrics

All the other details I’m interested in come from the MetricsClient. The metrics are striped by the type of storage, and that’s added to the resource ID when you build it. I’m interested in blob storage, so my resource ID is built from the account name plus “blob” – I’d use “table” or “queue” if I wanted metrics on the other storage types.

The MetricsClient is the new style API client; from Microsoft.WindowsAzure.Management.Monitoring version 4.0.0.0, you new-up the client object with a Microsoft.Azure.SubscriptionCloudCredentials object.

I can fetch the total number of blob requests by listing the “TotalRequests” metric, and that runs in hourly time-grains, so I can get the total for the last hour:

model.Metrics = new BlobMetric();
var resourceId = ResourceIdBuilder.BuildStorageResourceId(_serviceName, "blob");
var credentials = Subscription.GetCredentials();
using (var metricsClient = new MetricsClient(credentials))
{
    var requestMetrics = await metricsClient.MetricValues.ListAsync(resourceId, new[] { "TotalRequests" }, null, TimeSpan.FromHours(1), DateTime.UtcNow.AddHours(-1), DateTime.UtcNow);
    var totalMetric = requestMetrics.MetricValueSetCollection.Value.First();
    model.Metrics.LatestRequestCount = totalMetric.MetricValues.First(x => x.Total.HasValue).Total;
}

When you’re listing metrics you can pass an array of metric names, but you have to use a time grain and time span which is valid for all the metrics in the list. For blob storage, there’s a metric which records the number of successful requests called “Success”, which has the same time details as “TotalRequests” so I could include that in the same list call.

That’s what I do to get the total amount of storage used (misleadingly called ‘Capacity’), and the count of objects and containers, all of which  run on 1-day time grains:

var dayMetrics = await metricsClient.MetricValues.ListAsync(resourceId, new[] { "ContainerCount", "ObjectCount", "Capacity" }, null, TimeSpan.FromDays(1), DateTime.UtcNow, DateTime.UtcNow);

var usageMetric = dayMetrics.MetricValueSetCollection.Value.Single(x => x.Name == "Capacity");
model.Metrics.StorageUsed = usageMetric.MetricValues.First(x => x.Total.HasValue).Total;

var containerMetric = dayMetrics.MetricValueSetCollection.Value.Single(x => x.Name == "ContainerCount");
model.Metrics.ContainerCount = containerMetric.MetricValues.First(x => x.Total.HasValue).Total;

var objectMetric = dayMetrics.MetricValueSetCollection.Value.Single(x => x.Name == "ObjectCount");
model.Metrics.ObjectCount = objectMetric.MetricValues.First(x => x.Total.HasValue).Total;

For this and any other metrics you try and fetch, if you ask for an invalid combination of metric name and time details, or you ask for a metric which doesn’t exist for the service type, you’ll get an exception with an unhelpful message:

{“Code”:”InvalidRequest”, “Message”:”Fix the request and try again”".”}

If you make a valid request but the metrics aren’t being stored (you have to explicitly enable metric collection for storage accounts), or they haven’t been calculated yet (e.g. asking for object count during the first day when a storage account is created) you’ll get an empty MetricValueSetCollection.

Sample Data

For one of my storage accounts, which has a moderate amount of storage and usage, this is the data I get back from my populated model:

{
    "Redundancy": "Standard_GRS",
    "SecondaryLocation": "West Europe",
    "Metrics": {
        "ContainerCount": 4.0,
        "ObjectCount": 81.0,
        "LatestRequestCount": 474.0,
        "StorageUsed": 11682900829.0
    },
    "Headline": "Standard_GRS",
    "Label": "Secondary: West Europe",
    "Name": "mystorageaccount",
    "FriendlyName": null,
    "Status": "Created",
    "Location": "North Europe"
}

And now, the github link?

It’ll be in the next post, “Dashing Azure: Free, Simple Dashboards for your Azure Estate”. Coming soon.

Posted on Tuesday, July 14, 2015 11:25 AM Azure | Back to top


Comments on this post: Hey Azure, how's my Blob Storage doing?

No comments posted yet.
Your comment:
 (will show your gravatar)


Copyright © Elton Stoneman | Powered by: GeeksWithBlogs.net