HOWTO: Programmatically Accessing Web Content using Basic Authentication

Accessing data from a URL is a fairly straightforward exercise. Many services have implemented a REST style interface to allow queries to be submitted and the results returned to the client, usually as XML.

For example:

StringBuilder restURL = new StringBuilder; HttpWebRequest restRequest; HttpWebResponse restResponse; XmlDocument xDoc = new XmlDocument;

// build the URL String - dr is a DataReader // ADO.NET code omitted for clarity restURL.AppendFormat("http://myServer/rest/item?catalogNumber={0}&itemOwner={1}", dr["ItemNumber"], dr["ItemOwner"]);

// use the static Create method of the WebRequest object // casting the returned WebRequest to an HttpWebRequest restRequest = (HttpWebRequest) WebRequest.Create(restURL.ToString);

// use the GetResponse method to obtain a WebResponse object // for the request casting to an HttpWebResponse restResponse = (HttpWebResponse) restRequest.GetResponse;

// since we are expecting XML back, we can load the // XML document directly from the GetResponseStream // method of the HttpWebResponse xDoc.Load(restResponse.GetResponseStream);

// do something really cool with the Document:-)

Now, what if we had the following URL and that particular path was secured using Basic Authentication? For an in-depth look at Basic and Digest Authentication, or if you can't sleep, you can read RFC-2617 HTTP Authentication: Basic and Digest Access Authentication.

If we were to hit that link with a browser, we'd simply be presented with a logon dialog, fill in our credentials, and click OK. The problem is, we don't have a browser to gather our credetials and pass them back to the webserver. On top of that, the webserver is expecting the username and password to be Base64 encoded. Fortunately, everything we need to solve this dilema is right in the.NET Framework.

According to the RFC, we need to pass back the Authorization header in the following form: Authorization: Basic userid:password with the “userid:password“ portion Base64 encoded. Adding the Authorization header is easy. The HttpWebRequest object exposes the Headers collection and we can call the Add method of that collection passing in the name of the header we wish to add along with the value. So, to use the example given in the RFC, if we wish to pass back the userid “Aladin“ and the password “open sesame“ then we can do the following:

restRequest.Headers.Add("Authorization", "Basic " + "QWxhZGRpbjpvcGVuIHNlc2FtZQ==";

The gobbledy-goop on the end is actually the Base64 encoding of Aladin:open sesame. No need to panic. We can again turn to the.NET Framework to make this conversion simple. As one might expect, the System.Convert class contains the static method ToBase64String. ToBase64String expects an array of unsigned integers. What the? Why is nothing every simple?

Now we need to turn “Aladin:open sesame” into an array of unsigned integers. Fortunately, System.Encoding contains just the thing we need. The System.Encoding.ASCII.GetBytes method takes a string as a parameter and returns a byte array containing the encoded version of the string. We feed that into Convert.ToBase64String and we're ready to go. We end up with this:

restRequest.Headers.Add("Authorization", "Basic " + Convert.ToBase64String(Encoding.ASCII.GetBytes(“Aladin:open sesame“)));

We just add that line before the call to GetResponse and everything works out. We just need to take care of error handling and then we're all set.

Of course, we'll put all of this code inside a try{}catch{} block.

// catch web related exceptions

catch(WebException webEx)

{

      StringBuilder errorString = new StringBuilder;

      // Protocol errors are things like

      // 404 - Not Found

      // 401 - Unauthorized etc...

      if(webEx.Status == WebExceptionStatus.ProtocolError)

      {

            // we can get the actual status code from the original response

            // that is exposed through the Response property of the WebException class

            errorString.AppendFormat("Status Code: {0}", ((HttpWebResponse)webEx.Response).StatusCode);

            // the same goes for the description of the returned status

            errorString.AppendFormat("Status Description: {0}", ((HttpWebResponse)webEx.Response).StatusDescription);

      }

}

So, there you go. I hope that this will be of some value to you in the future.

Dave Just because I can... (living the hell so that you don't have to)

New on Geeks with Blogs