Shaun Xu

The Sheep-Pen of the Shaun


News

logo

Shaun, the author of this blog is a semi-geek, clumsy developer, passionate speaker and incapable architect with about 10 years experience in .NET. He hopes to prove that software development is art rather than manufacturing. He's into cloud computing platform and technologies (Windows Azure, Aliyun) as well as WCF and ASP.NET MVC. Recently he's falling in love with JavaScript and Node.js.

Currently Shaun is working at IGT Technology Development (Beijing) Co., Ltd. as the architect responsible for product framework design and development.

MVP

My Stats

  • Posts - 96
  • Comments - 344
  • Trackbacks - 0

Tag Cloud


Recent Comments


Recent Posts


Archives


Post Categories



On Nov 5th, Microsoft announced a new service named Windows Azure Scheduler, which allows us to invoke actions (such as calling HTTP/S endpoints or posting a message to a storage queue) on any schedule. Then on Dec 13th, Scott Guthrie published a blog post introduced this new feature. In this post Scott demonstrated how to use this feature to create a blog poll job each 5 minutes through the Windows Azure portal. At the end of his article he mentioned this feature can be used through .NET API package.

I think using Windows Azure Scheduler from our code would be more frequently than from the portal, since normally we might need define, update, enable or disable jobs from our application or through our application instead of asking our end users to update jobs from azure portal directly. So in this post I would like to introduce how to use this feature from a .NET code.

 

Windows Azure Scheduler Management Library on NuGet

Currently Windows Azure Scheduler SDK is still in preview phase on NuGet. So if we search the package from Visual Studio Manage NuGet Packages dialog we will not be able to find it.

Hence we must open the Package Manager Console window in Visual Studio and install it in command-line mode with the preview flag specified. Let's create a new console application and open the Package Manager Console then type of command as below. Do not forget the last "-Pre" argument.

image

 

Create Cloud Service

If you have read Scott's blog you should understand that in Windows Azure Scheduler each job will be defined in a job collection. But you might not know that each job collection should be located under a "Cloud Service". And more confused, this "Cloud Service" was totally a different entity than the "Cloud Service" we are familiar with which can be used to host web role, worker role and virtual machines.

In this case the "Cloud Service" is a container for resources intended to be more utilized in future versions of the service management API. And there are some resource providers that implement this new internal API such as Windows Azure store, HDInsight and some others. Windows Azure Scheduler is another resource that will be located in this "Cloud Service" so before we created job collection and job, we must firstly create a new Cloud Service, define the region where our scheduler will be hosted.

 

We will create a new cloud service through Windows Azure Scheduler SDK. First of all we need to populate a valid Windows Azure credentials with a management certificate configured properly. The code below retrieved the management certificate from local machine and initialized a certificate cloud credential instance by specifying its thumbprint.

Please refer this document on how to create, upload a Windows Azure Management Certificate. And make sure you have install this certificate on your machine at Current User.

   1: static void Main(string[] args)
   2: {
   3:     // retrieve the windows azure managment certificate from current user
   4:     var store = new X509Store(StoreName.My, StoreLocation.CurrentUser);
   5:     store.Open(OpenFlags.ReadWrite);
   6:     var certificate = store.Certificates.Find(X509FindType.FindByThumbprint, "{THE-CERTIFICATE-THUMBPRINT}", false)[0];
   7:     store.Close();
   8:  
   9: }

Then I created a subscription credential from this certificate and its Windows Azure Subscription ID.

   1: static void Main(string[] args)
   2: {
   3:     // retrieve the windows azure managment certificate from current user
   4:     ... ...
   5:  
   6:     // create management credencials and cloud service management client
   7:     var credentials = new CertificateCloudCredentials("SUBSCRIPTION-ID", certificate);
   8:     var cloudServiceMgmCli = new CloudServiceManagementClient(credentials);
   9:  
  10: }

Now we would be able to create a cloud service for our Windows Azure Scheduler from the management client. In the code below I created a new cloud service located at South Center US.

   1: static void Main(string[] args)
   2: {
   3:     // retrieve the windows azure managment certificate from current user
   4:     ... ...
   5:  
   6:     // create management credencials and cloud service management client
   7:     ... ...
   8:  
   9:     // create cloud service
  10:     var cloudServiceCreateParameters = new CloudServiceCreateParameters()
  11:     {
  12:         Description = "shaun-wasch-demo",
  13:         Email = "jfarrio@gmail.com",
  14:         GeoRegion = "South Central US",
  15:         Label = "shaun-wasch-demo"
  16:     };
  17:     var cloudService = cloudServiceMgmCli.CloudServices.Create("shaun-wasch-demo", cloudServiceCreateParameters);
  18: }

 

Create Job Collection and Job

Now we have a cloud service available, the next step is to create a job collection inside it. In the code below I initialized a new management client named "SchedulerManagementClient" which takes the responsible for managing the scheduler job collection as well as the resource provider.

You might need to invoke "SchedulerManagementClient.RegisterResourceProvider" to register a new resource provider if you got the exception "The subscription is not entitled to use the resource" when you executed the code listed below.

Ref the resource provider problem please refer to Sandrino Di Mattia's blog.

Once we have "SchedulerManagementClient" ready we can create a new job collection with the plan and quota specified. Since I used "Free" plan so the quote must be smaller than once per hour, and assigned to the cloud service I had just created.

   1: static void Main(string[] args)
   2: {
   3:     // retrieve the windows azure managment certificate from current user
   4:     ... ...
   5:  
   6:     // create management credencials and cloud service management client
   7:     ... ...
   8:  
   9:     // create cloud service
  10:     ... ...
  11:  
  12:     // create job collection
  13:     var schedulerMgmCli = new SchedulerManagementClient(credentials);
  14:     var jobCollectionIntrinsicSettings = new JobCollectionIntrinsicSettings()
  15:     {
  16:         Plan= JobCollectionPlan.Free,
  17:         Quota = new JobCollectionQuota()
  18:         {
  19:             MaxJobCount = 5,
  20:             MaxJobOccurrence = 1,
  21:             MaxRecurrence = new JobCollectionMaxRecurrence()
  22:             {
  23:                 Frequency = JobCollectionRecurrenceFrequency.Hour,
  24:                 Interval = 1
  25:             }
  26:         }
  27:     };
  28:     var jobCollectionCreateParameters=new JobCollectionCreateParameters()
  29:     {
  30:          IntrinsicSettings=jobCollectionIntrinsicSettings,
  31:          Label = "jc1"
  32:     };
  33:     var jobCollectionCreateResponse = schedulerMgmCli.JobCollections.Create("shaun-wasch-demo", "jc1", jobCollectionCreateParameters);
  34: }

Finally I will create a new job in this job collection. To manage jobs I need to initialize "SchedulerClient" which can be used to create, update and delete jobs inside a job collection. In this example I defined this job with a HTTP polling action to this blog website once an hour, with the job named "poll_blog".

   1: static void Main(string[] args)
   2: {
   3:     // retrieve the windows azure managment certificate from current user
   4:     ... ...
   5:  
   6:     // create management credencials and cloud service management client
   7:     ... ...
   8:  
   9:     // create cloud service
  10:     ... ...
  11:  
  12:     // create job collection
  13:     ... ...
  14:  
  15:     // create job
  16:     var schedulerClient = new SchedulerClient(credentials, "shaun-wasch-demo", "jc1");
  17:     var jobAction = new JobAction()
  18:     {
  19:         Type = JobActionType.Http,
  20:         Request = new JobHttpRequest()
  21:         {
  22:             Uri = new Uri("http://blog.shaunxu.me"),
  23:             Method = "GET"
  24:         }
  25:     };
  26:     var jobRecurrence = new JobRecurrence()
  27:     {
  28:         Frequency = JobRecurrenceFrequency.Hour,
  29:         Interval = 1
  30:     };
  31:     var jobCreateOrUpdateParameters = new JobCreateOrUpdateParameters()
  32:     {
  33:         Action = jobAction,
  34:         Recurrence = jobRecurrence
  35:     };
  36:     var jobCreateResponse = schedulerClient.Jobs.CreateOrUpdate("poll_blog", jobCreateOrUpdateParameters);
  37: }

You might notice that I invoked "SchedulerClient.Jobs.CreateOrUpdate" instead of "SchedulerClient.Jobs.Create". This because "CreateOrUpdate" will trigger an HTTP PUT request to Windows Azure Management RESTful API with job name provided, while "Create" will trigger an HTTP POST request that job name will be generated by Windows Azure automatically.

So if you utilized "SchedulerClient.Jobs.Create" you will find the job name would be an GUID.

Now we have finished all codes to create a cloud service, job collection and job from the .NET SDK. The full code listed below.

   1: using Microsoft.WindowsAzure;
   2: using Microsoft.WindowsAzure.Management.Scheduler;
   3: using Microsoft.WindowsAzure.Management.Scheduler.Models;
   4: using Microsoft.WindowsAzure.Scheduler;
   5: using Microsoft.WindowsAzure.Scheduler.Models;
   6: using System;
   7: using System.Collections.Generic;
   8: using System.Linq;
   9: using System.Security.Cryptography.X509Certificates;
  10: using System.Text;
  11: using System.Threading.Tasks;
  12:  
  13: namespace AzureSchedulerDemo
  14: {
  15:     class Program
  16:     {
  17:         static void Main(string[] args)
  18:         {
  19:             // retrieve the windows azure managment certificate from current user
  20:             var store = new X509Store(StoreName.My, StoreLocation.CurrentUser);
  21:             store.Open(OpenFlags.ReadWrite);
  22:             var certificate = store.Certificates.Find(X509FindType.FindByThumbprint, "MANAGEMENT-CERTIFICATE-THUMBPRINT", false)[0];
  23:             store.Close();
  24:  
  25:             // create management credencials and cloud service management client
  26:             var credentials = new CertificateCloudCredentials("SUBSCRIPTION-ID", certificate);
  27:             var cloudServiceMgmCli = new CloudServiceManagementClient(credentials);
  28:  
  29:             // create cloud service
  30:             var cloudServiceCreateParameters = new CloudServiceCreateParameters()
  31:             {
  32:                 Description = "shaun-wasch-demo",
  33:                 Email = "jfarrio@gmail.com",
  34:                 GeoRegion = "South Central US",
  35:                 Label = "shaun-wasch-demo"
  36:             };
  37:             var cloudService = cloudServiceMgmCli.CloudServices.Create("shaun-wasch-demo", cloudServiceCreateParameters);
  38:  
  39:             // create job collection
  40:             var schedulerMgmCli = new SchedulerManagementClient(credentials);
  41:             var jobCollectionIntrinsicSettings = new JobCollectionIntrinsicSettings()
  42:             {
  43:                 Plan= JobCollectionPlan.Free,
  44:                 Quota = new JobCollectionQuota()
  45:                 {
  46:                     MaxJobCount = 5,
  47:                     MaxJobOccurrence = 1,
  48:                     MaxRecurrence = new JobCollectionMaxRecurrence()
  49:                     {
  50:                         Frequency = JobCollectionRecurrenceFrequency.Hour,
  51:                         Interval = 1
  52:                     }
  53:                 }
  54:             };
  55:             var jobCollectionCreateParameters=new JobCollectionCreateParameters()
  56:             {
  57:                  IntrinsicSettings=jobCollectionIntrinsicSettings,
  58:                  Label = "jc1"
  59:             };
  60:             var jobCollectionCreateResponse = schedulerMgmCli.JobCollections.Create("shaun-wasch-demo", "jc1", jobCollectionCreateParameters);
  61:  
  62:             // create job
  63:             var schedulerClient = new SchedulerClient(credentials, "shaun-wasch-demo", "jc1");
  64:             var jobAction = new JobAction()
  65:             {
  66:                 Type = JobActionType.Http,
  67:                 Request = new JobHttpRequest()
  68:                 {
  69:                     Uri = new Uri("http://blog.shaunxu.me"),
  70:                     Method = "GET"
  71:                 }
  72:             };
  73:             var jobRecurrence = new JobRecurrence()
  74:             {
  75:                 Frequency = JobRecurrenceFrequency.Hour,
  76:                 Interval = 1
  77:             };
  78:             var jobCreateOrUpdateParameters = new JobCreateOrUpdateParameters()
  79:             {
  80:                 Action = jobAction,
  81:                 Recurrence = jobRecurrence
  82:             };
  83:             var jobCreateResponse = schedulerClient.Jobs.CreateOrUpdate("poll_blog", jobCreateOrUpdateParameters);
  84:  
  85:             Console.WriteLine("Press any key to exit");
  86:             Console.ReadKey();
  87:         }
  88:     }
  89: }

Now let's execute this application. This will invoke Windows Azure Management API to create the job collection and job. Then we can back to Windows Azure portal to see a new job collection had been created at the region we defined.

image

Click into this job collection we will find the plan and quota is the same as what we wanted.

image

And the job we created from our application will be shown in "Jobs" tab with the name we specified.

image

And we will find that this job had been executed at least once after we submitted from the "History" tab.

image

If we back to the "Dashboard" tab of this job collection, we will find the URI display at the right side of the page. The cloud service was shown inside the URI as below.

image

 

View Job History and More

Besides the job creation, we can retrieve the job execution history through this SDK.

   1: // retrieve job history
   2: var schedulerClient = new SchedulerClient(credentials, "shaun-wasch-demo", "jc1");
   3: var jobGetHistoryParameters = new JobGetHistoryParameters()
   4: {
   5:     Skip = 0,
   6:     Top = 100
   7: };
   8: var history = schedulerClient.Jobs.GetHistory("poll_blog", jobGetHistoryParameters);
   9: foreach(var action in history)
  10: {
  11:     Console.WriteLine("{0}\t{1}\t{2}\t{3}\t{4}", action.Status, action.Message, action.RetryCount, action.RepeatCount, action.Timestamp);
  12: }

The history will be displayed in the command window.

image

Which the same what we saw in Windows Azure portal.

image

We can also retrieve the job collections from a given cloud service, jobs from a given job collection. And we can update, delete job collections and jobs as well.

When define a job, we could be able define the retry policy and error action of the job executed failed. This two parameter can only be specified through .NET SDK.

 

Summary

Windows Azure Scheduler allows us to define a time-based jobs in Windows Azure platform. We can use this feature to invoke a website or web service through HTTP request regularly. We can also let the job insert message into a Windows Azure Storage Queue so that some background worker roles can be trigger to start some backend processes, and this might be more useful in our application design and development.

For example in one of my project there will be some backend jobs which retrieve some data from some web services and insert into Windows Azure SQL Database, and some other jobs will be started hourly or monthly to process those data. In my original deign, all jobs will be stored in Windows Azure Table and I utilized a dedicate worker role to check if it's time to start a job, generate a message into a queue then a related worker role will handle this message and start process the job. In this mode I need to implement how to store jobs (action and recurrence settings) into Table, how to go though the table and calculate when a job should be started. I also had to deal with concurrency if more than one worker roles to check the jobs.

image

Now by using the Windows Azure Scheduler I can simplify my design as below. Those stuff such as how to store the jobs definition, how to start a job cloud be leave to Windows Azure Scheduler. What I need to do is to implement the relevant business logic for each job.

image

 

Hope this helps,

Shaun

All documents and related graphics, codes are provided "AS IS" without warranty of any kind.
Copyright © Shaun Ziyan Xu. This work is licensed under the Creative Commons License.

Comments

No comments posted yet.
Post A Comment
Title:
Name:
Email:
Comment:
Verification: