Grooming Windows Azure Diagnostics Storage and IIS Logs

People working with Windows Azure are aware that the storage used for diagnostics will continue to grow perpetually if nothing is done about it.

With the introduction of the Windows Azure Management Pack – I call it WAMP; don’t know if it has an official acronym yet – System Center Operations Manager (SCOM) is able to groom the tables.

By default the following three rules are disabled in WAMP:

  • Windows Azure Role Performance Counter Grooming
  • Windows Azure Role .NET Trace Grooming
  • Windows Azure Role NET Event Log Grooming
    Once they have been enabled the rule will run on a periodic basis and will delete data from the relevant table older than T hours.

An online guide to WAMP is available here.

Unfortunately WAMP/SCOM does not come with a similar functionality to groom the IIS logs located in blob storage. By default these logs are written to the blob storage once every hour, so after a few months in production there are quite a number of them. And remember, that it is one log entry per instance.

The cost of storage is not very big, so it would be difficult to argue for an automated solution to groom the logs if price is the only parameter considered. However, as the number of entries grow it will take longer and longer to actually identify the relevant one.

To overcome this challenge one can write a small PowerShell script using e.g. the Azure Management Cmdlets developed by Cerebrata.

The script could look like the following:

# Name of your account
$accountName = <account name>
# Account key
$accountKey = <acount key># Name of container, e.g. wad-iis-logfiles
$containerName = <Container Name>
$lastModified = <UTC Date>

$bfc = New-Object Cerebrata.AzureUtilities.ManagementClient.StorageManagementEntities.BlobsFilterCriteria

$bfc.LastModifiedDateTimeTo = $lastModified

Remove-Blob -BlobContainerName $containerName
-BlobsFilterCriteria $bfc
-AccountName $accountName
-AccountKey $accountKey

The script will remove all blob entries older than the date given.

About strobaek

.NET developer/architect. Runner, espresso drinker and lover of gourmet food.
This entry was posted in Azure and tagged . Bookmark the permalink.

One Response to Grooming Windows Azure Diagnostics Storage and IIS Logs

  1. Pingback: Windows Azure and Cloud Computing Posts for 9/8/2011+ - Windows Azure Blog

Leave a Reply

Your email address will not be published. Required fields are marked *