Shaun Xu

The Sheep-Pen of the Shaun


News

logo

Shaun, the author of this blog is a semi-geek, clumsy developer, passionate speaker and incapable architect with about 10 years experience in .NET. He hopes to prove that software development is art rather than manufacturing. He's into cloud computing platform and technologies (Windows Azure, Aliyun) as well as WCF and ASP.NET MVC. Recently he's falling in love with JavaScript and Node.js.

Currently Shaun is working at IGT Technology Development (Beijing) Co., Ltd. as the architect responsible for product framework design and development.

MVP

 

I’m not pretty sure how many users are using the standard SQL Server Management Studio (SSMS) 2008 R2 to manage your SQL Azure database. I was one of them until I found the Houston project provided by Microsoft in its SQL Azure Labs.

SQL Azure Labs is a place where we can try the latest CTP features of the SQL Azure. They might not be included in the production SQL Azure but I think most of them will be launched in the future. So it’s a good place for us to play with the newest stuff of SQL Azure.

image

As you can see there are three features now provided in the labs and in this post I would like to introduce the 3rd one – Project Houston CTP 1.

 

Limitation of SSMS 08 R2

I was using the SSMS 08 R2 to access my SQL Azure database before. It works, but not that pretty. There are some limitation I found when I used it.

  • I can’t use the standard design view to create and modify my tables. It seems that the SQL Azure platform doesn’t support the SQL Server Configuration Service or something else. So what I can do is to use the plain SQL statement. Although I can connect to my local mirror database to finished my modification and them copy and run the generated SQL to SQL Azure it still very inefficiency.
  • If the one above is acceptable for you since the tables schema changing was not that frequently, it should be very frustrated that the SSMS doesn't support listing the data of a SQL Azure table. This means I have to write the query manually when I need to check the rows.
  • Since the data traffic should be charged by Microsoft each time I fetch or insert data I need to pay for them.

image

 

Houston CTP 1

The project Houston is a web-based, in fact it’s Silverlight-based application which you can manage you SQL Azure database with more features than the SSMS.

The registration is very simple. You firstly navigate to the Project Houston CTP 1 section on the SQL Azure Labs with your Live ID and then you need to select a location where the Houston will be run. In order to maximize the performance you’d better select the same location that hosts your SQL Azure database. In my case I selected the North Central US.

image

And then the connection and logon form appeared. Here I said “form” rather than “page” is because we have been in the Silverlight environment. Just provide the SQL Azure database server name, database name and your account here.

Note: You need to specify your full database server address in the Server box. You can get the server address in your SQL Azure Development Portal.

image

 

After logged into the Houston you can see the basic information of your SQL Azure server and database includes the users, connections, versions and the quota.

image

image

 

On the left there are the tables, views and stored procedures you’ve created in the database and you can create, update the schema through its design view, which SSMS doesn’t support.

image

And just click the Data button on the top you can see the rows in a grid where you can add, update and delete them which SSMS doesn’t support neither.

image

Also we can add, update our stored procedures although it doesn’t not support the IntellSense. But I think it would support in the future.

image

 

Summary

In this post I just introduced a lightweight but powerful tool for managing the SQL Azure database. It’s web-based which means we don’t need to install but it’s very rich by using Silverlight.

While the Azure platform become more popular, more and more tools are available that can be used by us to improve the development efficiency.

 

Hope this helps,

Shaun

 

All documents and related graphics, codes are provided "AS IS" without warranty of any kind.
Copyright © Shaun Ziyan Xu. This work is licensed under the Creative Commons License.

 

Long time didn’t post any articals about the Windows Azure since I was fully busy with our first cloud-based platform – XLR8 (codename: Xalent). Just a week ago my chief architect Ray asked me to try to deploy our platform on Windows Azure. There should something need to be changed and one of them is, on Azure we cannot use the local file system to store anything which was uploaded by the end users. There are two reasons:

  • All files under the web role project will be treated as one package. This means when we deployed a web role the Azure will delete the original web application folder and files then extract our new package and initialize it. Hence all files uploaded by the users will be deleted at this moment.
  • In some cases the Windows Azure platform may migrate your application from one virtual machine to another. But we cannot assume that the root path of our application will be the same. So if we use Server.Mappath() it might return different result.

Hence when our application is running on Azure the uploaded files would better to be stored into the blob storage service.

 

Challenge and Purpose

If we want to migrate a normal web application to azure-based application we need to modify all the codes for uploading the files, and maybe the code for displaying the image files as well. But now what I’m facing with is that, the web application should be satisfied with azure and normal deployment at same time. This means it’s unacceptable to change the file operation code in the business logic and UI when it’s deployed to azure and normal server. We need to make sure the code can work well on both deployment and what we can change should only be the configuration.

One of the approach is that we can use Cloud Drive feature. In that way we can mount a VHD file located at BLOB storage as a local hard disk. Then it's almost no need to change the IO operations and codes. But storing the files in BLOB storage directly make bring some other benefits such as, it would be possible to access these files through URL directly. In this post I would like to introduce another way.

So the challenge now is that, I need a partten to take the responsible for operating files regardless if it uses normal file system, or the blob storage service. In this post I will explain a bit about what I did and hopefully it could be useful for you when you develop the azure and normal web application in the future.

 

Simple Architecture and Implementation

The architecture is very simple. In order to make the web application depends on the abstract file operations I created an interface to isolate the implementation between the file system and blob service.

image

In the IFileSystemAgent interface I defined the basic file operation methods such as Save, Load, Delete and Exists. The GetResourceUrl method is to be used to return the URL when we need to link the file, especially display the image file, on a web page. It could return the proper URL based on which system we are using now.

   1: public interface IFileSystemAgent
   2: {
   3:     void Save(Stream fileStream, string filename, bool overwrite);
   4:  
   5:     void Save(byte[] bytes, string filename, bool overwrite);
   6:  
   7:     byte[] Load(string filename);
   8:  
   9:     bool Exists(string filename);
  10:  
  11:     void Delete(string filename);
  12:  
  13:     string GetResourceUrl(string filename);
  14: }

Under the IFileSystemAgent I implemented two classes one for windows file system which can be used for normal deployment, and the other for windows azure which can be used for blob storage service.

 

The main different between these two implementation classes was not only the file operations, but how to deal with the root path as well. As we know when using the windows file system in a web application we normally use Server.MapPath() to translate the virtual path to a physical path and save or load the file. But in blob storage we need to retrieve the storage account information and upload the bytes or stream to the endpoint of the storage account. This is very different.

And when we need to display or link the files on a web page, in windows file system we just need to use the relevant path for example “/upload/images/beijing-hotel-img1_50x50.jpg”. But in blob storage the path should be like “http://xlr8.blob.core.windows.net/default/beijing-hotel-img1_50x50.jpg”.

Hence when saving and linking a file, the IFileSystemAgent only accept the file name with the relevant path, and the implementation class will determine where and how to save or link it.

 

In the WindowsFileSystemAgent I passed the HttpServerUtilityBase through its constructor and also a parameter named Root which means the files will be stored at Server.MapPath(“/” + Root) dirctionary. And In the AzureBlobFileSystemAgent I passed the CloudStorageAccount and the ContainerName through the constructor so that the files will be stored at that blob account within that container.

Below are the implementation code for these two classes.

   1: public class WindowsFileSystemAgent : IFileSystemAgent
   2: {
   3:     private HttpServerUtilityBase _server;
   4:     private string _root;
   5:  
   6:     public HttpServerUtilityBase Server
   7:     {
   8:         get
   9:         {
  10:             return _server;
  11:         }
  12:         set
  13:         {
  14:             _server = value;
  15:         }
  16:     }
  17:  
  18:     public string Root
  19:     {
  20:         get
  21:         {
  22:             return _root;
  23:         }
  24:         set
  25:         {
  26:             _root = value;
  27:         }
  28:     }
  29:  
  30:     public WindowsFileSystemAgent()
  31:         : this(null, string.Empty)
  32:     {
  33:     }
  34:  
  35:     public WindowsFileSystemAgent(HttpServerUtilityBase server, string root)
  36:     {
  37:         _server = server;
  38:         _root = root;
  39:     }
  40:  
  41:     private string GetServerSideFullname(string filename)
  42:     {
  43:         return Path.Combine(_server.MapPath("/" + _root), filename);
  44:     }
  45:  
  46:     #region IFileSystemAgent Members
  47:  
  48:     public void Save(Stream fileStream, string filename, bool overwrite)
  49:     {
  50:         byte[] bytes = new byte[fileStream.Length];
  51:         fileStream.Read(bytes, 0, (int)fileStream.Length);
  52:  
  53:         Save(bytes, filename, overwrite);
  54:     }
  55:  
  56:     public void Save(byte[] bytes, string filename, bool overwrite)
  57:     {
  58:         filename = GetServerSideFullname(filename);
  59:         var directory = Path.GetDirectoryName(filename);
  60:         if (!Exists(directory))
  61:         {
  62:             Directory.CreateDirectory(directory);
  63:         }
  64:         if (Exists(filename))
  65:         {
  66:             if (overwrite)
  67:             {
  68:                 Delete(filename);
  69:             }
  70:             else
  71:             {
  72:                 throw new ApplicationException(string.Format("Existed file {0} please select another name or set the overwrite = true."));
  73:             }
  74:         }
  75:         using (var stream = File.Create(filename))
  76:         {
  77:             stream.Write(bytes, 0, bytes.Length);
  78:         }
  79:     }
  80:  
  81:     public byte[] Load(string filename)
  82:     {
  83:         filename = GetServerSideFullname(filename);
  84:         byte[] bytes;
  85:         using (var stream = File.OpenRead(filename))
  86:         {
  87:             bytes = new byte[stream.Length];
  88:             stream.Read(bytes, 0, bytes.Length);
  89:         }
  90:         return bytes;
  91:     }
  92:  
  93:     public bool Exists(string filename)
  94:     {
  95:         filename = GetServerSideFullname(filename);
  96:         if (File.Exists(filename))
  97:         {
  98:             return true;
  99:         }
 100:         else
 101:         {
 102:             return Directory.Exists(filename);
 103:         }
 104:     }
 105:  
 106:     public void Delete(string filename)
 107:     {
 108:         filename = GetServerSideFullname(filename);
 109:         if (File.Exists(filename))
 110:         {
 111:             File.Delete(filename);
 112:         }
 113:     }
 114:  
 115:     public string GetResourceUrl(string filename)
 116:     {
 117:         return "/" + _root + "/" + filename;
 118:     }
 119:  
 120:     #endregion
 121: }
   1: public class AzureBlobFileSystemAgent : IFileSystemAgent
   2: {
   3:     private static string CST_DEFAULTCONTAINERNAME = "default";
   4:     private static string CST_DEFAULTACCOUNTSETTING = "DataConnectionString";
   5:  
   6:     private string _containerName { get; set; }
   7:     private CloudStorageAccount _storageAccount { get; set; }
   8:  
   9:     private CloudBlobContainer _container;
  10:  
  11:     public AzureBlobFileSystemAgent()
  12:         : this(CST_DEFAULTCONTAINERNAME, CST_DEFAULTACCOUNTSETTING)
  13:     {
  14:     }
  15:  
  16:     public AzureBlobFileSystemAgent(string containerName, string storageAccountConnectionString)
  17:         : this(containerName, CloudStorageAccount.FromConfigurationSetting(storageAccountConnectionString))
  18:     {
  19:     }
  20:  
  21:     public AzureBlobFileSystemAgent(string containerName, CloudStorageAccount storageAccount)
  22:     {
  23:         _containerName = containerName;
  24:         _storageAccount = storageAccount;
  25:  
  26:         // create the blob container for account logos if not exist
  27:         CloudBlobClient blobStorage = _storageAccount.CreateCloudBlobClient();
  28:         _container = blobStorage.GetContainerReference(_containerName);
  29:         _container.CreateIfNotExist();
  30:  
  31:         // configure blob container for public access
  32:         BlobContainerPermissions permissions = _container.GetPermissions();
  33:         permissions.PublicAccess = BlobContainerPublicAccessType.Container;
  34:         _container.SetPermissions(permissions);
  35:     }
  36:  
  37:     #region IFileSystemAgent Members
  38:  
  39:     public void Save(Stream fileStream, string filename, bool overwrite)
  40:     {
  41:         var bytes = new byte[fileStream.Length];
  42:         fileStream.Read(bytes, 0, bytes.Length);
  43:  
  44:         Save(bytes, filename, overwrite);
  45:     }
  46:  
  47:     public void Save(byte[] bytes, string filename, bool overwrite)
  48:     {
  49:         filename = TranslateFileName(filename);
  50:         CloudBlockBlob blob = _container.GetBlockBlobReference(filename);
  51:         if (Exists(filename))
  52:         {
  53:             if (overwrite)
  54:             {
  55:                 Delete(filename);
  56:             }
  57:             else
  58:             {
  59:                 throw new ApplicationException(string.Format("Existed file {0} please select another name or set the overwrite = true."));
  60:             }
  61:         }
  62:         blob.UploadByteArray(bytes, new BlobRequestOptions() { Timeout = TimeSpan.FromMinutes(3) });
  63:     }
  64:  
  65:     public byte[] Load(string filename)
  66:     {
  67:         filename = TranslateFileName(filename);
  68:         CloudBlockBlob blob = _container.GetBlockBlobReference(filename);
  69:         return blob.DownloadByteArray();
  70:     }
  71:  
  72:     public bool Exists(string filename)
  73:     {
  74:         filename = TranslateFileName(filename);
  75:         CloudBlockBlob blob = _container.GetBlockBlobReference(filename);
  76:         try
  77:         {
  78:             blob.FetchAttributes();
  79:             return true;
  80:         }
  81:         catch (StorageClientException ex)
  82:         {
  83:             if (ex.ErrorCode == StorageErrorCode.ResourceNotFound)
  84:             {
  85:                 return false;
  86:             }
  87:             else
  88:             {
  89:                 throw;
  90:             }
  91:         }
  92:     }
  93:  
  94:     public void Delete(string filename)
  95:     {
  96:         filename = TranslateFileName(filename);
  97:         CloudBlockBlob blob = _container.GetBlockBlobReference(filename);
  98:         blob.DeleteIfExists();
  99:     }
 100:  
 101:     private string TranslateFileName(string filename)
 102:     {
 103:         return filename.Replace('/', '~').Replace('\\', '`');
 104:     }
 105:  
 106:     public string GetResourceUrl(string filename)
 107:     {
 108:         // when using the local storage simulator the blob enpoint without the end '/'
 109:         // but when using the azure it has '/' at the end of it
 110:         // so here i have to use Path.Combine to construct the path and then replace the '\' back to '/'
 111:         var url = Path.Combine(_storageAccount.BlobEndpoint.ToString(), _containerName, TranslateFileName(filename));
 112:         return url.Replace('\\', '/');
 113:     }
 114:  
 115:     #endregion
 116: }

 

Save and Display Image in ASP.NET MVC

Let me use an ASP.NET MVC application to explain how to use it. First of all we need a helper class to initialize the instance of the IFileSystemAgent based on the configuration. Here I just created a very simple factory class to return the instance based on the value in the web.config. But if in a real project we’d better use some IoC containers to do this such as Unity.

   1: public static class FileSystemAgentFactory
   2: {
   3:     public static IFileSystemAgent Resolve()
   4:     {
   5:         var config = System.Configuration.ConfigurationManager.AppSettings["filesystem-agent"];
   6:         switch(config.ToLower())
   7:         {
   8:             case "windows":
   9:                 if (HttpContext.Current != null && HttpContext.Current.Server != null)
  10:                 {
  11:                     return new WindowsFileSystemAgent(new HttpServerUtilityWrapper(HttpContext.Current.Server), "Upload");
  12:                 }
  13:                 else
  14:                 {
  15:                     throw new NotSupportedException("HttpContext ot its Server property is null. The WindowsFileSystemAgent must be used under the web application.");
  16:                 }
  17:             case "blob":
  18:                 return new AzureBlobFileSystemAgent();
  19:             default:
  20:                 return null;
  21:         }
  22:     }
  23: }

And then in a controller which handles the file upload we can use this factory to initialize the proper implementation instance of our IFileSystemAgent and just call its Save method to save the file regardless which one we are actually using now. And if we need to migrate our application between the azure and normal server we just need to modify the web.config file.

   1: [HttpPost]
   2: public ActionResult UploadFile(string filekey)
   3: {
   4:     if (Request.Files != null && Request.Files.Count > 0)
   5:     {
   6:         var file = Request.Files[0];
   7:         var filename = "Avatar/" + Guid.NewGuid().ToString() + Path.GetExtension(file.FileName);
   8:  
   9:         var filesys = FileSystemAgentFactory.Resolve();
  10:         filesys.Save(file.InputStream, filename, true);
  11:  
  12:         Repository.Images.Add(filename);
  13:     }
  14:  
  15:     return RedirectToAction("Index");
  16: }

 

Similarly when we need to display or link a file on a web page we also no need to consider where it’s being stored. In order for this I created an extension method on the HtmlHelper. With this helper method when we need to display or link a file we just use the GetResourceUrl method of the IFileSystemAgent so it will return the proper URL back.

   1: public static class HelpHelpers
   2: {
   3:     public static MvcHtmlString Image(this HtmlHelper helper, string filename)
   4:     {
   5:         return Image(helper, FileSystemAgentFactory.Resolve(), filename);
   6:     }
   7:  
   8:     public static MvcHtmlString Image(this HtmlHelper helper, IFileSystemAgent agent, string filename)
   9:     {
  10:         return Image(helper, agent, filename, VirtualPathUtility.GetFileName("/" + filename));
  11:     }
  12:  
  13:     public static MvcHtmlString Image(this HtmlHelper helper, IFileSystemAgent agent, string filename, string alt)
  14:     {
  15:         var html = string.Format("<img src=\"{0}\" alt=\"{1}\" />", agent.GetResourceUrl(filename), alt);
  16:         return MvcHtmlString.Create(html);
  17:     }
  18: }

 

Summary

I this post I explained a bit about my solution on how to make the file operation code unified in our application especially a website between azued-based and normal deployment. There are still something can be improved in my solution. One is that we can separate the HttpServerUtilityBase and the CloudStorageAccount as an interface. For example, IRootProvider, so that they can be injected and can be fully unit tested.

There also some other points can be improved for azure application migration. For example in azure we’d better put the frequently change configuration values to the ServiceConfiguration.cscfg file rather than the web.config. This lead us to build a provider for loading the configuration which I’m going to explain in the future.

 

Note: You can download the full demo code here.

 

Hope this helps,

Shaun

 

All documents and related graphics, codes are provided "AS IS" without warranty of any kind.
Copyright © Shaun Ziyan Xu. This work is licensed under the Creative Commons License.

 


 

Just found this information at the Windows Azure Team Blog. But it only for the US developers. (Why! Geo-location Discrimination?) Anyway, it’s said the first 500 would get it. For more information please go here.

 

Hope this helps,

Shaun

All documents and related graphics, codes are provided "AS IS" without warranty of any kind.
Copyright © Shaun Ziyan Xu. This work is licensed under the Creative Commons License.