Jesse Taber

Enumerations And The CStr Function

Anyone who has worked with Visual Basic for any length of time should be familiar with the VB Type Conversion Functions like CBool, CInt, and CStr. These functions can provide a short and easy way to coerce the value of one type into another. I’ve been working with ~300,000 line VB .NET project for the past couple of years that is littered with calls to these functions. We never had any issues with them until we sat down to convert the project to C#.

Converting a 300,000 line project from VB .NET to C# is a pretty big undertaking and I plan on putting together a post-mortem at some point in the future, but I wanted to briefly outline one subtle issue we hit with the CStr function and enumerations.

Consider the following enumeration (declared in C#):

1     public enum ReviewStatus
2     {
3         NotStarted = 0,
4         InProgress,
5         Complete
6     }

Let’s say you want to use this enumeration in the query string of a web page that displays a report. In VB .NET you could easily use the CStr function to take an instance of the ReviewStatus enumeration and convert it to a string:

1 Dim selectedStatus As ReviewStatus = GetSelectedReviewStatus()
2 Request.QueryString.Add("reviewStatus", CStr(selectedReviewStatus))

This results in a URL that looks like:

/somePage.aspx?reviewStatus=1

When provided with a Numeric Data Type the CStr function will return a string representing the number. The use of CStr here makes me a bit nervous because it doesn’t provide any means of specifying an IFormatInfo, but if globalization isn’t a concern using CStr here is probably “good enough”.

When we set out to convert our project to C# we found dozens of places where we were using CStr to convert an enumeration into a string. Because there is no direct equivalent of CStr in C#, we opted to use ToString instead. The VB .NET code above turned into:

1 var selectedStatus = GetSelectedReviewStatus();
2 Request.Querystring.Add("reviewStatus", selectedStatus.ToString());

At first we didn’t see any issues with this conversion until we started testing the application. We soon discovered that the URL in the converted C# project looked like this:

/somePage.aspx?reviewStatus=InProgress

The code that consumes the ‘reviewStatus’ query string variable was expecting an integer that could be cast to the appropriate enumeration member was was throwing exceptions trying to parse the ‘InProgress’ value.

To resolve this issue, we ended up having to first cast the enumeration value to an integer and then calling ToString on the newly cast integer.

But What About Convert.ToString ?

If you search Google for the phrase, “CStr equivalent in C#” you’re likely to find forum posts indicating that Convert.ToString should be used in place of CStr. Convert.ToString might be an appropriate substitute for CStr in many instances, but using Convert.ToString on an enumeration in C# will yield the same result as .ToString’ing the enumeration value, and therefore didn’t help with our problem.

• Share This Post:
• 
• 
• 

Posted On Tuesday, January 24, 2012 9:11 PM | Feedback (0) |

‘Content-Length’ Header Replaced With ‘Transfer-Encoding: chunked’ in ASP .NET

About a week ago we received a support call from one of our customers asking why she couldn’t open PDF files on our website anymore. She was using Chrome and the browser tab would just hang with the ‘Loading’ animation and a gray background after requesting an invoice PDF. After a little digging we figured out that this hang-up was occurring while Chrome was opening the PDF document. We were easily able to reproduce this issue ourselves in the production environment, but only while using Chrome. We were not, however, able to reproduce it in any of our QA / test environments. All environments are running the same version of ASP .NET (3.5) and IIS (7.5).

After trying dozens of different combinations of browser settings, PDF file sizes, and IIS settings, I finally noticed that the content-length header that we set explicitly before writing the PDF file bytes out the response stream was being replaced with a ‘transfer-encoding: chunked’ header in our production environment (but not in any of our QA environments). In some circumstances Chrome chokes on file downloads using the transfer-encoding: chunked header. Initial research into the issue yielded a post reporting that ASP .NET will set the transfer-encoding: chunked header for you when calling ‘Flush’ manually without explicitly setting a content-length header. Unfortunately for me we were explicitly setting the content-length header, so it was back to the drawing board.

After banging my head against the wall for a few hours comparing the raw IIS configuration XML between our QA and production servers I finally remembered that we had seen some very odd response related issues that stemmed from an HTTP Response Filter we were using on some pages. Sure enough, the filter was the cause of our issues in this instance as well.

The reason we were only seeing this issue in the production environment was that we use an HTTP Module to profile page requests. One of the things we measure is the size of the ‘viewstate’ field that is sent down for each request to detect pages that might be viewstate abusers. In order to track the size of the viewstate data sent down we attach a stream to the ‘Filter’ property of the HttpResponse that intercepts and records the value of the viewstate hidden field. Disabling this viewstate tracking filter for the page that generates and sends PDF files down to the browser resolved the issue. This module was enabled in our production environment but nowhere else.

It seems unlikely that anyone else will ever stumble across this same issue, but if I can save someone else a few hours of troubleshooting time with this blog post then it’ll be well worth the short time it took me to write this up. In summary, the following circumstances lead to the replacement of the content-length header with a transfer-encoding: chunked header:

• Site is hosted on IIS 7.5 (I never tested this on IIS 7, though I do know that the Visual Studio development web server does NOT exhibit the same issue)
• A stream is set on the ‘Filter’ property of the HttpResponse. What the filter actually does seems immaterial. I was able to reproduce the problem with a filter that simply passed the raw bytes through without modifying them at all.
• A content-length header is explicitly set.
• ‘Flush’ is called on the HttpResponse directly after writing some or all of the file contents to the response. (in our application we were calling Flush multiple times sending down about 1kb of data at a time as our files can be quite large sometimes.)
• (Optional) Calling ‘End’ on the HttpResponse after sending all of the file contents. (This is not explicitly required for seeing the transfer-encoding: chunked header, but this does seem to be a requisite for having Chrome choke on opening the PDF.

I created a quick and dirty web forms app to reproduce the issue that you can grab on GitHub. I was also able to deploy and reproduce the same issue using this sample app on AppHarbor:

• Example code on GitHub: https://github.com/appakz/TransferEncodingTest
• Example app on AppHarbor: http://transferencodingtest.apphb.com/

• Share This Post:
• 
• 
• 

Posted On Monday, January 02, 2012 7:58 PM | Feedback (0) |

Time Warp

It’s no secret that daylight savings time can wreak havoc on systems that rely heavily on dates. The system I work on is centered around recording dates and times, so naturally my co-workers and I have seen our fair share of date-related bugs. From time to time, however, we come across something that we haven’t seen before. A few weeks ago the following error message started showing up in our logs:

“The supplied DateTime represents an invalid time. For example, when the clock is adjusted forward, any time in the period that is skipped is invalid.”

This seemed very cryptic, especially since it was coming from areas of our application that are typically only concerned with capturing date-only (no explicit time component) from the user, like reports that take a “start date” and “end date” parameter. For these types of parameters we just leave off the time component when capturing the date values, so midnight is used as a “placeholder” time. How is midnight an “invalid time”?

Globalization Is Hard

Over the last couple of years our software has been rolled out to users in several countries outside of the United States, including Brazil. Brazil begins and ends daylight savings time at midnight on pre-determined days of the year. On October 16, 2011 at midnight many areas in Brazil began observing daylight savings time at which time their clocks were set forward one hour. This means that at the instant it became midnight on October 16, it actually became 1:00 AM, so any time between 12:00 AM and 12:59:59 AM never actually happened.

Because we store all date values in the database in UTC, always adjust any “local” dates provided by a user to UTC before using them as filters in a query. The error we saw was thrown by .NET when trying to convert the Brazilian local time of 2011-10-16 12:00 AM to UTC since that local time never actually existed. We hadn’t experienced this same issue with any of our US customers because the daylight savings time changes in the US occur at 2:00 AM which doesn’t conflict with our “placeholder” time of midnight.

Detecting Invalid Times

In .NET you might use code similar to the following for converting a local time to UTC:

  1: var localDate = new DateTime(2011, 10, 16); //2011-10-16 @ midnight
  2: const string timeZoneId = "E. South America Standard Time"; //Windows system timezone Id for "Brasilia" timezone.
  3: var localTimeZone = TimeZoneInfo.FindSystemTimeZoneById(timeZoneId);
  4: var convertedDate =  TimeZoneInfo.ConvertTimeToUtc(localDate, localTimeZone);

The code above throws the “invalid time” exception referenced above. We could try to detect whether or not the local time is invalid with something like this:

  1: var localDate = new DateTime(2011, 10, 16); //2011-10-16 @ midnight
  2: const string timeZoneId = "E. South America Standard Time"; //Windows system timezone Id for "Brasilia" timezone.
  3: var localTimeZone = TimeZoneInfo.FindSystemTimeZoneById(timeZoneId);
  4: if (localTimeZone.IsInvalidTime(localDate))
  5:   localDate = localDate.AddHours(1);
  6: var convertedDate =  TimeZoneInfo.ConvertTimeToUtc(localDate, localTimeZone);

This code works in this particular scenario, but it hardly seems robust. It also does nothing to address the issue that can arise when dealing with the ambiguous times that fall around the end of daylight savings. When we roll the clocks back an hour they record the same hour on the same day twice in a row. To continue on with our Brazil example, on February 19, 2012 at 12:00 AM, it will immediately become February 18, 2012 at 11:00 PM all over again. In this scenario, how should we interpret February 18, 2011 11:30 PM?

Enter Noda Time

I heard about Noda Time, the .NET port of the Java library Joda Time, a little while back and filed it away in the back of my mind under the “sounds-like-it-might-be-useful-someday” category.  Let’s see how we might deal with the issue of invalid and ambiguous local times using Noda Time (note that as of this writing the samples below will only work using the latest code available from the Noda Time repo on Google Code. The NuGet package version 0.1.0 published 2011-08-19 will incorrectly report unambiguous times as being ambiguous) :

  1: var localDateTime = new LocalDateTime(2011, 10, 16, 0, 0); 
  2: const string timeZoneId = "Brazil/East";
  3: var timezone = DateTimeZone.ForId(timeZoneId);
  4: var localDateTimeMaping = timezone.MapLocalDateTime(localDateTime);
  5:  
  6: ZonedDateTime unambiguousLocalDateTime;
  7: switch (localDateTimeMaping.Type)
  8: {
  9:   case ZoneLocalMapping.ResultType.Unambiguous:
 10:     unambiguousLocalDateTime = localDateTimeMaping.UnambiguousMapping;
 11:     break;
 12:   case ZoneLocalMapping.ResultType.Ambiguous:
 13:     unambiguousLocalDateTime = localDateTimeMaping.EarlierMapping;
 14:     break;
 15:   case ZoneLocalMapping.ResultType.Skipped:
 16:     unambiguousLocalDateTime = new ZonedDateTime(
 17:     localDateTimeMaping.ZoneIntervalAfterTransition.Start, timezone);
 18:     break;
 19:   default:
 20:     throw new InvalidOperationException(
 21:       string.Format("Unexpected mapping result type: {0}", 
 22:                      localDateTimeMaping.Type));
 23: }
 24: DateTime convertedDateTime = unambiguousLocalDateTime.ToInstant().ToDateTimeUtc();

Let’s break this sample down:

• If the provided local date time is unambiguous in the provided time zone I can immediately set the ‘unambiguousLocalDateTime’ variable from the ‘Unambiguous Mapping’ property of the mapping returned by the ‘MapLocalDateTime’ method.
• If the provided local date time is ambiguous in the provided time zone (i.e. it falls in an hour that was repeated when moving clocks backward from Daylight Savings to Standard Time), I can use the ‘EarlierMapping’ property to get the earlier of the two possible local dates to define the unambiguous local date and time that I need. I could have also opted to use the ‘LaterMapping’ property in this case, or even returned an error and asked the user to specify the proper choice. The important thing to note here is that as the programmer I’ve been forced to deal with what appears to be an ambiguous date and time.
• If the provided local date time represents a skipped time (i.e. it falls in an hour that was skipped when moving clocks forward from Standard Time to Daylight Savings Time),  I have access to the time intervals that fell immediately before and immediately after the point in time that caused my date to be skipped. In this case I have opted to disambiguate my local date and time by moving it forward to the beginning of the interval immediately following the skipped period. Again, I could opt to use the end of the interval immediately preceding the skipped period, or raise an error depending on the needs of the application.

This sample is admittedly contrived and could certainly use some refactoring, but I think it captures the general approach needed to take a local date and time and convert it to UTC with Noda Time. At first glance it might seem that Noda Time makes this “simple” code more complicated and verbose because it forces you to explicitly deal with the local date disambiguation, but I feel that the length and complexity of the Noda Time sample is proportionate to the complexity of the problem. Using TimeZoneInfo leaves you susceptible to overlooking ambiguous and skipped times that could result in run-time errors or (even worse) run-time data corruption in the form of a local date and time being adjusted to UTC incorrectly.

I should point out that this research is my first look at Noda Time and I know that I’ve only scratched the surface of its full capabilities. I also think it’s safe to say that it’s still beta software for the time being so I’m not rushing out to use it production systems just yet, but I will definitely be tinkering with it more and keeping an eye on it as it progresses.

• Share This Post:
• 
• 
• 

Posted On Tuesday, November 15, 2011 9:30 PM | Feedback (0) |

Parsing Concerns–Part 2

In a previous post I explored some of the issues that can arise when parsing date and time values provided by users from different cultures and regions of the world. Correctly parsing and formatting different date formats is a very common internationalization concern in most software projects, but in this post I want to explore another area where parsing and formatting can cause some headaches.

Numbers Are The International Language, Right?

If your application deals with quantities of any kind then you probably have to both accept numeric input from users and later display that numeric input back to the user. Consider the following numbers:

• 2.782

• 7,482

What do these numbers represent to you? If you’re from the United States you’d probably interpret these numbers as:

• 2.782 = a decimal representing two whole units plus 782/1000 partial units

• 7,482 = an integer (whole number) representing seven thousand four hundred and eighty two whole units

.NET software running on my servers in the United States would interpret those numbers the same way:

//default current culture to en-US 

Thread.CurrentThread.CurrentCulture = new CultureInfo("en-US");             
var firstNumber = decimal.Parse("2.782");

//must explicitly allow for the thousands separator

var secondNumber = int.Parse("7,482", NumberStyles.AllowThousands); 

The code above runs successfully and parses both numeric values without any loss of precision. So what happens if this same code is running under a culture other than en-US?

Thread.CurrentThread.CurrentCulture = new CultureInfo("de-DE");
var firstNumber = decimal.Parse("2.782");
var secondNumber = int.Parse("7,482", NumberStyles.AllowThousands);

Now that the culture has been changed to de-DE (German),something interesting happens. The number formatting rules for the de-DE culture in .NET specify that the decimal symbol is a comma and the thousands separator is a period (basically the opposite of the en-US rules). When I was first experimenting with number parsing in an internationalized application I would have expected both of these parse calls to throw an exception due to improper formatting. As I stated in my last post about parsing, a thrown exception would be a good thing in this case, as it would call attention to some parsing code that is possibly not ready to deal with values from different cultures. Unfortunately in this case, the first parsing call does not throw an exception. Instead, the first call parses the value “2.782” and interprets the “.” as a thousands separator. The resulting .NET decimal value is now two-thousand seven-hundred and eighty-two meaning that the originally intended value has been corrupted and lost.

How Do You Deal With This?

The moral of my last post about parsing and formatting boiled down to: “Know Your Audience”, and this post about parsing and formatting is no different. It’s critical to know where numeric values are coming from and culture the source of the values belongs to.  It also helps to use the most precise parsing logic that you can so that there is little room for ambiguity. In the example above, if we use the decimal.Parse overload that accepts a NumberStyles enumeration value and provide NumberStyles.Float, the call to parse running under the German culture will fail, as it will be expecting a proper decimal separator. It’s easy to ignore all of those “extraneous” parameters to the Parse and Format methods in .NET, but they really can have a big impact on how your code behaves. When in doubt, look it up or fire up a quick LINQpad session and test out some different variations. If there’s a silver-bullet solution to problems like this I certainly haven’t found it and every application/scenario likely calls for a slightly different approach. The point of post was to call attention to an oddity of decimal parsing that can cause issues in multi-cultural applications.

What’s Next?

While being more restrictive in parsing logic is good for eliminating ambiguity and mitigating the risk of data being improperly parsed, it can lead to a poor user experience. Disallowing the use of thousands separators or using a consistent decimal separator across all cultures is a bit user-hostile (and I’ve certainly been guilty of it). In a future post I want to explore some strategies for allowing users to enter the same data in a variety of ways that make sense to them while still maintaining strict parsing and validation.

• Share This Post:
• 
• 
• 

Posted On Sunday, October 02, 2011 1:15 PM | Feedback (0) |

Validating Input In An Internationalized Application

If your application accepts input from users you’re likely going to want to perform some validation on that input. If you allow users to create data within your system that is displayed to other users (i.e. it should be “human-readable”) you might consider implementing a validation routine that ensures that all input for this data consist only of letters and whitespace. While I generally tend to shy away from overly aggressive validation of user input, I think that this kind of validation can make sense in some scenarios where you wouldn’t want someone’s cat running across their keyboard to create a document titled “((@(8#jk(@(‘’299” in your system.

The Simple Solution

But how should you go about this validation? A simple regular expression (is there such a thing?) should do the trick, right? A quick Google search for “regex for letters and whitespace” might yield something like this: ^[a-zA-Z ]+$ (loosely based on the question and accepted answer of a Stack Overflow question). In short, this regular expression would match any string consisting of the letters A-Z (upper or lower case) and a white space.

This regex would allow the user to enter “My Document”, but not “ZOMG The. Best. Document. EVERRRR!!!!!!11!!1one :-)”. This is great and does what you want until your application needs to support someone who doesn’t speak English. (Side note: I normally wouldn’t care if someone wanted to name a document something silly. This is an admittedly contrived example that I put together to illustrate a point. Be nice to your users and always validate responsibly.)

Your Alphabet Is Not The Only Alphabet

The alphabets used in other languages (if they even have an alphabet) often make use of diacritical marks. In the Czech alphabet, for example, there use some familiar looking letters like ‘A’, ‘E’, and ‘J’, but there are also letters like ‘Č’ and ‘Ů’. It’s important to note here that the diacritical marks above the ‘Č’ and ‘Ů’ are not just “accent marks”. That is to say that they aren’t just modifiers that are applied to the “normal” ‘C’ and ‘U’ letters; they are completely different letters.

So when a user goes to create a document called ‘Můj dokument’ (which is likely an atrocious translation of ‘My Document’… my apologies to the Czechs), your regular expression will reject it. You could be an arrogant American and tell them to just make it ‘Muj dokument’, but that’s not very nice. The point of your validation routine was to help ensure that input remained human-readable, and using characters in foreign alphabets is a perfectly human thing to do.

In this particular scenario you’d likely be better suited using the regular expression shorthand for “word character”, which is “\w”. In the .NET flavor of regex, this will allow for Unicode letters, meaning letters with diacritics are fair game. It’s important to note that it also allows digits (0-9) and the underscore character but not whitespace. In my opinion, the digits 0-9 and the underscore are perfectly acceptable things to allow in a human-readable string, but you could always modify your validation routine to expressly check for digits and underscores if there’s some really good reason to disallow them.

Why Should I Care?

If your application is and will always be for a target audience of English-speaking users, then maybe you don’t need to care. If you think there’s any remote chance that your application might someday be used by folks that don’t speak English, do yourself a favor and take a second to think about how best to validate input from you users.

• Share This Post:
• 
• 
• 

Posted On Tuesday, June 07, 2011 9:33 PM | Feedback (0) |

WebImage.GetImageFromRequest() returns null

TL;DR: The WebImage.GetImageFromRequest() is case-sensitive with regard to the extensions of the image file names. More info here: http://forums.asp.net/t/1682158.aspx/1

Lately I’ve been working on a little side project in my spare time that requires that I allow users to upload an image from their computer. I’ve been using ASP .NET MVC 3 with Razor views and have been having a great time with it so far.

I was excited to find the microsoft-web-helpers package on NuGet, as it contains what appeared to be a really useful helper class called ‘WebImage’ for easily accepting uploaded images and performing simple manipulation like resizing and cropping. I was happily using this little library for awhile before discovering that is has a few pretty significant issues. First of all, the resize functionality completely breaks under certain circumstances, but that issue is beyond the scope of this post. The issue that really put the breaks on my momentum was much more fundamental: the ‘GetImageFromRequest’ method, which is used for grabbing an image from the POSTed form data, was returning ‘null’ for seemingly valid images.

I had the basic functionality of this little application more or less done and had been testing it locally my own computer without issue, but had also recruited some friends and family to try it out as well. I was getting really odd reports that it wasn’t working for some pictures that were taken on a digital camera. This was really puzzling and I spent hours pouring over the properties of the jpeg files that were failing, but couldn’t find any smoking gun.

Today I finally did what I should have done days ago and did a little Googling to see if this was a known issue. Within a couple of minutes I had found a post on the ASP .NET forums that explained the issue: the ‘GetImageFromRequest’ method has problems if the extension of the image file name is capitalized (e.g. ‘GIF’ instead of ‘gif’). When the images were being saved to the computer from the digital camera they were coming across with .JPG extensions, which was causing the issue. Thankfully there are a couple of good suggestions on that forum post with good workarounds. I ended up creating a substitute ‘GetImageFromRequest’ method as advocated in one of the responses which seems to be working great so far.

I’ve been a bit disappointed in the WebImage helper class. These issues I’ve encountered really prevent me from using it in the way I wanted to and, in my opinion, make it unusable for any “real” application. I really hope that these issues get addressed in a future version.

• Share This Post:
• 
• 
• 

Posted On Monday, May 30, 2011 1:51 PM | Feedback (0) |

Color Code Your SSMS Windows With the SSMS Toolpack

If your typical workweek is anything like mine you might have connections to multiple SQL Server instances open in SQL Server Management Studio at any given time. For me, some of these instances are on my local machine (i.e. I use them for development work), and some of them are on remote (read: test or production) machines. In a perfect world I’d never need to jump into a production database to investigate an issue, but the world I live in is far from perfect so I end up looking at “live” databases from time to time.

Because I can sometimes have multiple connections going at once I always have a little voice in the back of my head asking, “Am I about to run this query against the wrong instance?” I usually like to keep connections to remote machines open only for as long as I need them to help mitigate that risk but in my imperfect world sometimes I get distracted by a phone call or IM. I’ve done a pretty good job of conditioning myself to take a quick peak at the status bar at the bottom of my SSMS query window to make sure that the server says what I think it should before kicking off any query that will alter data or cause locking.

I was doing some screen-sharing with a co-worker the other day and noticed that he had a little colored bar at the top of his query window. Turns out that he was using the Window Connection Coloring feature in the SSMS Toolpack (which is a great add-in for SQL Server Management Studio). This feature lets you have a color-coded bar present in every query window that changes based on what server that window is pointed at.

If you have the SSMS Toolpack installed just go to the “SSMS Tools –> Window Connection Coloring –> Options” menu to bring up the dialog for configuring your connection highlight settings. I set up mine to look something like this:

Rather than setting up a different entry for every server I connect to, I setup two entries:

• One for ‘localhost’ that uses a green connection color.
• One using a regex for any server name that isn’t ‘localhost’ that uses a red connection color. The regex I used is: ^((?!localhost).*)$ (also be sure to check the ‘IsRegex’ column).

This dialog UI is a bit wonky, so make sure that you click the ‘Save’ button once you get everything the way that you want it before clicking ‘OK’.

With these settings I now have a bright green bar at the top of every query window that is pointed at ‘localhost’ and a bright red one that is pointed at every other server I connect to. Now I know that I don’t want those red windows open for very long!

Happy Green Window (localhost):

Scary Red Window (remote machine):

• Share This Post:
• 
• 
• 

Posted On Tuesday, April 05, 2011 8:34 PM | Feedback (1) |

A Quick Look At AppHarbor

As mentioned in some earlier posts, I’ve been hacking on Shopify for the past six months for an e-commerce project. My work with Shopify introduced me to Heroku, which is a simple “cloud-based” service for hosting and running Ruby applications. It’s dead simple to get started and, for the most part, seems to “just work”.

While I enjoy playing with new languages and platforms, I’m by no means a Ruby developer. Heroku is a great service, but seemed somewhat inaccessible to me given that I’d have to work with an unfamiliar language in order to use it. It seems that Microsoft’s Azure platform is trying to fill this same niche for .NET developers, but my (admittedly limited) experiences with Azure have been far from smooth.

When I heard AppHarbor being referred to as a “Heroku For .NET / Azure Done Right” I headed over to their site and signed up for an account right away. I’ve been playing around with it for the past several days to see if it lives up to the hype. In short: I’m pretty impressed with what I’ve seen so far. This post outlines some of my initial findings.

Setup

The AppHarbor site is fairly minimalist in its design, which makes it very easy to get started. Upon landing on their home page visitors are immediately greeted with the message: “.NET Deployment Done Right … git push appharbor master”. Awesome.

If you’ve used a service like github before then getting up and running on AppHarbor will feel very natural. If you’ve never used github or git before it’ll take a minute to grok, but is completely worth it. For the rest of this post I’ll assume at least a basic familiarity with using git and the concept of remote repositories. There are tons of excellent resources out there for learning about git (I recommend the TekPub git series), so I won’t bother repeating that information here.

After signing up for a free account, head over to the msysgit project page to grab the latest and greatest version. I use git for some side projects and had a few issues trying to push to AppHarbor initially that were cleared up after upgrading to the latest version.

Creating And Deploying a ‘Hello World’ Application

Since I was digging into a new hosting platform I decided to go ahead and dig into the latest MVC 3 bits at the same time. The Web Platform Installer got me everything that I needed and I had a new boiler plate MVC 3 application up and running locally within a few minutes.

From here, initialize a new git repo in the solution folder where this solution and web project live. AppHarbor requires that the code you push to them contains a single solution file with a single web project (note that you can have other class library projects within that same solution for encapsulating code). I do most of my development this way so this approach is fine with me, but I did notice at least one thread in their support forum asking for multi-solution support. If I were them, this wouldn’t be at the top of my priority list for new features, so I have a feeling the single solution approach is probably going to stick for awhile.

Once you’ve initialized the git repo, added all of your files, and committed them to your local master repo, you’re ready to setup things on the AppHarbor side. Log into your AppHarbor account and go to the ‘Applications’ screen. Creating a new application is as easy as providing an application name. After creating the application you’re provided with a few quick steps for getting your code deployed and running in the AppHarbor environment.

Jump back into git and run the following command against your local master repo to create AppHarbor as a remote repository for your application (note that the git endpoint for your application can be found on the application details screen in AppHarbor):

git remote add appharbor <your application endpoint>

Now, deploying your application is as easy as:

git push appharbor master

You’ll be prompted for a password (which is the same as your AppHarbor account password) and provided with a status message informing you that your application has been queued for building. If you check the application page in AppHarbor you’ll be able to check on the status of the build:

When the build completes successfully, the code is automatically deployed and available to be run at your application’s URL (http://<app_name>.apphb.com).

You can view past builds and deployments and even choose to deploy an earlier build at any time, though if you had a database attached to your application that quick rollback could have some adverse side effects.

You can also view the details of each build and even get detailed console output to troubleshoot issues if they arise.

Environment Specifics

If you’re like me, you prefer having your entire development stack run locally when doing development work. This means that you like having your IDE, light-weight web server, and database all play nicely together on your development machine when you’re cranking out new features. This is great, but the configuration you use when developing locally is almost never the same configuration you need when the application is being used or tested by others.

AppHarbor has a built-in mechanism for allowing your code to determine which environment it’s running in and change configuration settings accordingly. All you have to do is add the following line to the ‘appSettings’ section of your web.config file:

<add key=”Environment” value=”Debug”/>

After you deploy your application, this value is automatically set to the value “Release”. Similarly, you can add the same line to the app.config file in your unit test project and it will have it’s value set to “Test” while AppHarbor runs your unit tests (more on that in the next section).

In the past I’ve maintained separate configuration files for each environment I work with and had my build process choose the right file for the environment being targeted. I don’t currently see any way to make that work with AppHarbor, so in a future post I’ll be digging into some potential alternatives. I’m thinking about defining a separate assembly in my solution that will be responsible for managing all of the various environment specific configuration values (e.g. connection strings) and return the proper value to the calling code based on the ‘environment’ value found in the main configuration file.

Unit Tests

Outside of being able to use git to deploy my applications, the other thing that really piqued my interest in AppHarbor is their ability to automatically run unit tests immediately following each build prior to deploying the application.

To make use of this feature, all you need to do is add a unit test project to your application and reference one of the supported unit test framework libraries. As of this writing, recent versions of nUnit and xUnit were supported.

AppHarbor finds these references and automatically runs all of the tests contained in those assemblies prior to deploying your code. If all of the tests pass, the code is automatically deployed. If one or more tests failed, the application remains on the last successful build.

Potential

While I’ve really only scratched the surface, I see a lot of potential for AppHarbor as a hosting platform. I love their convention-based approach to hosting and deploying applications: one solution with a web project, automatically running tests for any assembly that references a test framework, automatically grabbing and changing the ‘Environment’ application setting, etc. These conventions make it very easy to get up and running. Furthermore, these are all conventions that, for the most part, make a lot of sense. I haven’t seen anything about AppHarbor that would force you to write your application in a way that would prevent it from being run on a dedicated, semi-dedicated, virtual, or shared hosting environment with no changes. To me, that makes choosing this platform a lot more attractive than some of the alternatives out there. I plan on digging in a bit deeper in a future post to explore some more advanced concerns like file system storage (if that’s even an option), working with databases, and session state.

• Share This Post:
• 
• 
• 

Posted On Wednesday, January 26, 2011 8:40 PM | Feedback (1) |

Going Paperless

One year ago I came to work for a company where the entire development team is 100% “remote”; we’re spread over 3 time zones and each of us works from home. This seems to be an increasingly popular way for people to work and there are many articles and blog posts out there enumerating the advantages and disadvantages of working this way. I had read a lot about telecommuting before accepting this job and felt as if I had a pretty decent idea of what I was getting into, but I’ve encountered a few things over the past year that I did not expect. Among the most surprising by-products of working from home for me has been a dramatic reduction in the amount of paper that I use on a weekly basis.

Hoarding In The Workplace

Prior to my current telecommute job I worked in what most would consider pretty traditional office environments. I sat in cubicles furnished with an enormous plastic(ish) modular desks, had a mediocre (at best) PC workstation, and had ready access to a seemingly endless supply of legal pads, pens, staplers and paper clips. The ready access to paper, countless conference room meetings, and abundance of available surface area on my desk and in drawers created a perfect storm for wasting paper. I brought a pad of paper with me to every meeting I ever attended, scrawled some brief notes, and then tore that sheet off to keep next to my keyboard to follow up on any needed action items. Once my immediate need for the notes was fulfilled, that sheet would get shuffled off into a corner of my desk or filed away in a drawer “just in case”. I would guess that for all of the notes that I ever filed away, I might have actually had to dig up and refer to 2% of them (and that’s probably being very generous). That said, on those rare occasions that I did have to dig something up from old notes, it was usually pretty important and I ended up being very glad that I saved them. It was only when I would leave a job or move desks that I would finally gather all those notes together and take them to shredding bin to be disposed of. When I left my last job the amount of paper I had accumulated over my three years there was absurd, and I knew coworkers who had substance-abuse caliber paper wasting addictions that made my bad habit look like nail-biting in comparison.

A Product Of My Environment

I always hated using all of this paper, but simply couldn’t bring myself to stop. It would look bad if I showed up to an important conference room meeting without a pad of paper. What if someone said something profound! Plus, everyone else always brought paper with them. If you saw someone walking down the hallway with a pad of paper in hand you knew they must be on their way to a conference room meeting. Some people even had fancy looking portfolio notebook sheaths that gave their legal pads all the prestige of a briefcase. No one ever worried about running out of fresh paper because there was an endless supply, and there certainly was no shortage of places to store and file used paper. In short, the traditional office was setup for using tons and tons of paper; it’s baked into the culture there. For that reason, it didn’t take long for me to kick the paper habit once I started working from home.

In my home office, desk and drawer space are at a premium. I don’t have the budget (or the tolerance) for huge modular office furniture in my spare bedroom. I also no longer have access to a bottomless pit of office supplies stock piled in cabinets and closets. If I want to use some paper, I have to go out and buy it. Finally (and most importantly), all of the meetings that I have to attend these days are “virtual”. We use instant messaging, VOIP, video conferencing, and e-mail to communicate with each other. All I need to take notes during a meeting is my computer, which I happen to be sitting right in front of all day. I don’t have any hard numbers for this, but my gut feeling is that I actually take a lot more notes now than I ever did when I worked in an office. The big difference is I don’t have to use any paper to do so. This makes it far easier to keep important information safe and organized.

The Right Tool For The Job

When I first started working from home I tried to find a single application that would fill the gap left by the pen and paper that I always had at my desk when I worked in an office. Well, there are no silver bullets and I’ve evolved my approach over time to try and find the best tool for the job at hand. Here’s a quick summary of how I take notes and keep everything organized.

• Notepad++ – This is the first application I turn to when I feel like there’s some bit of information that I need to write down and save. I use Launchy, so opening Notepad++ and creating a new file only takes a few keystrokes. If I find that the information I’m trying to get down requires a more sophisticated application I escalate as needed.
• The Desktop – By default, I save every file or other bit of information to the desktop. Anyone who has ever had to fix their parents computer before knows that this is a dangerous game (any file my mother has ever worked on is saved directly to the desktop and rarely moves anywhere else). I agree that storing things on the desktop isn’t a great long term approach to keeping organized, which is why I treat my desktop a bit like my e-mail inbox. I strive to keep both empty (or as close to empty as I possibly can). If something is on my desktop, it means that it’s something relevant to a task or project that I’m currently working on. About once a week I take things that I’m not longer working on and put them into my ‘Notes’ folder.
• The ‘Notes’ Folder – As I work on a task, I tend to accumulate multiple files associated with that task. For example, I might have a bit of SQL that I’m working on to gather data for a new report, a quick C# method that I came up with but am not yet ready to commit to source control, a bulleted list of to-do items in a .txt file, etc. If the desktop starts to get too cluttered, I create a new sub-folder in my ‘Notes’ folder. Each sub-folder’s name is the current date followed by a brief description of the task or project. Then all files related to that task or project go into that sub folder. By using the date as the first part of the folder name, these folders are automatically sorted in reverse chronological order. This means that things I worked on recently will generally be near the top of the list. Using the built-in Windows search functionality I now have a pretty quick and easy way to try and find something that I worked on a week ago or six months ago.
• Dropbox – Dropbox is a free service that lets you store up to 2GB of files “in the cloud” and have those files synced to all of the different computers that you use. My ‘Notes’ folder lives in Dropbox, meaning that it’s contents are constantly backed up and are always available to me regardless of which computer I’m using. They also have a pretty decent iPhone application that lets you browse and view all of the files that you have stored there. The free 2GB edition is probably enough for just storing notes, but I also pay $99/year for the 50GB storage upgrade and keep all of my music, e-books, pictures, and documents in Dropbox. It’s a fantastic service and I highly recommend it.
• Evernote – I use Evernote mostly to organize information that I access on a fairly regular basis. For example, my Evernote account has a running grocery shopping list, recipes that my wife and I use a lot, and contact information for people I contact infrequently enough that I don’t want to keep them in my phone. I know some people that keep nearly everything in Evernote, but there’s something about it that I find a bit clunky, so I tend to use it sparingly.
• Google Tasks – One of my biggest paper wasting habits was keeping a running task-list next to my computer at work. Every morning I would sit down, look at my task list, cross off what was done and add new tasks that I thought of during my morning commute. This usually resulted in having to re-copy the task list onto a fresh sheet of paper when I was done. I still keep a running task list at my desk, but I’ve started using Google Tasks instead. This is a dead-simple web-based application for quickly adding, deleting, and organizing tasks in a simple checklist style. You can quickly move tasks up and down on the list (which I use for prioritizing), and even create sub-tasks for breaking down larger tasks into smaller pieces.
• Balsamiq Mockups – This is a simple and lightweight tool for creating drawings of user interfaces. It’s great for sketching out a new feature, brainstorm the layout of a interface, or even draw up a quick sequence diagram. I’m terrible at drawing, so Balsamiq Mockups not only lets me create sketches that other people can actually understand, but it’s also handy because you can upload a sketch to a common location for other team members to access.

I can honestly say that using these tools (and having limited resources at home) have lead me to cut my paper usage down to virtually none. If I ever were to return to a traditional office workplace (hopefully never!) I’d try to employ as many of these applications and techniques as I could to keep paper usage low. I feel far less cluttered and far better organized now.

• Share This Post:
• 
• 
• 

Posted On Sunday, January 09, 2011 8:15 PM | Feedback (0) |

Parsing Concerns

If you’ve ever written an application that accepts date and/or time inputs from an external source (a person, an uploaded file, posted XML, etc.) then you’ve no doubt had to deal with parsing some text representing a date into a data structure that a computer can understand. Similarly, you’ve probably also had to take values from those same data structure and turn them back into their original formats. Most (all?) suitably modern development platforms expose some kind of parsing and formatting functionality for turning text into dates and vice versa. In .NET, the DateTime data structure exposes ‘Parse’ and ‘ToString’ methods for this purpose. This post will focus mostly on parsing, though most of the examples and suggestions below can also be applied to the ToString method.

The DateTime.Parse method is pretty permissive in the values that it will accept (though apparently not as permissive as some other languages) which makes it pretty easy to take some text provided by a user and turn it into a proper DateTime instance. Here are some examples (note that the resulting DateTime values are shown using the RFC1123 format):

DateTime.Parse("3/12/2010"); //Fri, 12 Mar 2010 00:00:00 GMT
DateTime.Parse("2:00 AM");  //Sat, 01 Jan 2011 02:00:00 GMT (took today's date as date portion)
DateTime.Parse("5-15/2010"); //Sat, 15 May 2010 00:00:00 GMT
DateTime.Parse("7/8"); //Fri, 08 Jul 2011 00:00:00 GMT
DateTime.Parse("Thursday, July 1, 2010"); //Thu, 01 Jul 2010 00:00:00 GMT

Dealing With Inaccuracy

While the DateTime struct has the ability to store a date and time value accurate down to the millisecond, most date strings provided by a user are not going to specify values with that much precision. In each of the above examples, the Parse method was provided a partial value from which to construct a proper DateTime. This means it had to go ahead and assume what you meant and fill in the missing parts of the date and time for you. This is a good thing, especially when we’re talking about taking input from a user. We can’t expect that every person using our software to provide a year, day, month, hour, minute, second, and millisecond every time they need to express a date. That said, it’s important for developers to understand what assumptions the software might be making and plan accordingly. I think the assumptions that were made in each of the above examples were pretty reasonable, though if we dig into this method a little bit deeper we’ll find that there are a lot more assumptions being made under the covers than you might have previously known.

One of the biggest assumptions that the DateTime.Parse method has to make relates to the format of the date represented by the provided string. Let’s consider this example input string: ‘10-02-15’. To some people. that might look like ‘15-Feb-2010’. To others, it might be ‘02-Oct-2015’. Like many things, it depends on where you’re from.

This Is America!

Most cultures around the world have adopted either a “little-endian” or “big-endian” date format. (Source: Date And Time Notation By Country) In this context,  a “little-endian” date format would list the date parts with the least significant first while the “big-endian” date format would list them with the most significant first. For example, a “little-endian” date would be shown as “day-month-year” and “big-endian” would be “year-month-day”. It’s worth nothing here that ISO 8601 defines a “big-endian” format as the international standard.

While I personally prefer “big-endian” style date formats, I think both styles make sense in that they follow some logical standard with respect to ordering the date parts by their significance. Here in the United States, however, we buck that trend by using what is, in comparison, a completely nonsensical format of “month/day/year”. Almost no other country in the world uses this format. I’ve been fortunate in my life to have done some international travel, so I’ve been aware of this difference for many years, but never really thought much about it. Until recently, I had been developing software for exclusively US-based audiences and remained blissfully ignorant of the different date formats employed by other countries around the world.

The web application I work on is being rolled out to users in different countries, so I was recently tasked with updating it to support different date formats. As it turns out, .NET has a great mechanism for dealing with different date formats right out of the box. Supporting date formats for different cultures is actually pretty easy once you understand this mechanism.

Pulling the Curtain Back On the Parse Method

Have you ever taken a look at the different flavors (read: overloads) that the DateTime.Parse method comes in? In it’s simplest form, it takes a single string parameter and returns the corresponding DateTime value (if it can divine what the date value should be). (Note: many of you probably use the TryParse method. While the method overloads for TryParse are slightly different, the underlying behavior of that method with respect to date formats is the same). You can optionally provide two additional parameters to this method: an ‘System.IFormatProvider’ and a ‘System.Globalization.DateTimeStyles’. Both of these optional parameters have some bearing on the assumptions that get made while parsing a date, but for the purposes of this article I’m going to focus on the ‘System.IFormatProvider’ parameter.

The IFormatProvider exposes a single method called ‘GetFormat’ that returns an object to be used for determining the proper format for displaying and parsing things like numbers and dates. This interface plays a big role in the globalization capabilities that are built into the .NET Framework. The cornerstone of these globalization capabilities can be found in the ‘System.Globalization.CultureInfo’ class. To put it simply, the CultureInfo class is used to encapsulate information related to things like language, writing system, and date formats for a certain culture. Support for many cultures are “baked in” to the .NET Framework and there is capacity for defining custom cultures if needed (thought I’ve never delved into that). While the details of the CultureInfo class are beyond the scope of this post, so for now let me just point out that the CultureInfo class implements the IFormatInfo interface. This means that a CultureInfo instance created for a given culture can be provided to the DateTime.Parse method in order to tell it what date formats it should expect. So what happens when you don’t provide this value? Let’s crack this method open in Reflector:

When no IFormatInfo parameter is provided (i.e. we use the simple DateTime.Parse(string) overload), the ‘DateTimeFormatInfo.CurrentInfo’ is used instead. Drilling down a bit further we can see the implementation of the DateTimeFormatInfo.CurrentInfo property:

From this property we can determine that, in the absence of an IFormatProvider being specified, the DateTime.Parse method will assume that the provided date should be treated as if it were in the format defined by the CultureInfo object that is attached to the current thread (Thread.CurrentThread.CurrentCulture). The culture specified by the CultureInfo instance on the current thread can vary depending on several factors, but if you’re writing an application where a single instance might be used by people from different cultures (i.e. a web application with an international user base), it’s important to know what this value is. Having a solid strategy for setting the current thread’s culture for each incoming request in an internationally used ASP .NET application is obviously important, and might make a good topic for a future post. For now, let’s think about what the implications of not having the correct culture set on the current thread.

Let’s say you’re running an ASP .NET application on a server in the United States. The server was setup by English speakers in the United States, so it’s configured for US English. It exposes a web page where users can enter order data, one piece of which is an anticipated order delivery date. Most users are in the US, and therefore enter dates in a ‘month/day/year’ format. The application is using the DateTime.Parse(string) method to turn the values provided by the user into actual DateTime instances that can be stored in the database. This all works fine, because your users and your server both think of dates in the same way. Now you need to support some users in South America, where a ‘day/month/year’ format is used. The best case scenario at this point is a user will enter March 13, 2011 as ‘25/03/2011’. This would cause the call to DateTime.Parse to blow up since that value doesn’t look like a valid date in the US English culture (Note: In all likelihood you might be using the DateTime.TryParse(string) method here instead, but that method behaves the same way with regard to date formats). “But wait a minute”, you might be saying to yourself, “I thought you said that this was the best case scenario?” This scenario would prevent users from entering orders in the system, which is bad, but it could be worse! What if the order needs to be delivered a day earlier than that, on March 12, 2011? Now the user enters ‘12/03/2011’. Now the call to DateTime.Parse sees what it thinks is a valid date, but there’s just one problem: it’s not the right date. Now this order won’t get delivered until December 3, 2011. In my opinion, that kind of data corruption is a much bigger problem than having the Parse call fail.

What To Do?

My order entry example is a bit contrived, but I think it serves to illustrate the potential issues with accepting date input from users. There are some approaches you can take to make this easier on you and your users:

• Eliminate ambiguity by using a graphical date input control. I’m personally a fan of a jQuery UI Datepicker widget. It’s pretty easy to setup, can be themed to match the look and feel of your site, and has support for multiple languages and cultures.
• Be sure you have a way to track the culture preference of each user in your system. For a web application this could be done using something like a cookie or session state variable.
• Ensure that the current user’s culture is being applied correctly to DateTime formatting and parsing code. This can be accomplished by ensuring that each request has the handling thread’s CultureInfo set properly, or by using the Format and Parse method overloads that accept an IFormatProvider instance where the provided value is a CultureInfo object constructed using the current user’s culture preference.
• When in doubt, favor formats that are internationally recognizable. Using the string ‘2010-03-05’ is likely to be recognized as March, 5 2011 by users from most (if not all) cultures.
• Favor standard date format strings over custom ones. So far we’ve only talked about turning a string into a DateTime, but most of the same “gotchas” apply when doing the opposite. Consider this code:

someDateValue.ToString("MM/dd/yyyy");

This will output the same string regardless of what the current thread’s culture is set to (with the exception of some cultures that don’t use the Gregorian calendar system, but that’s another issue all together). For displaying dates to users, it would be better to do this:

someDateValue.ToString("d");

This standard format string of “d” will use the “short date format” as defined by the culture attached to the current thread (or provided in the IFormatProvider instance in the proper method overload). This means that it will honor the proper month/day/year, year/month/day, or day/month/year format for the culture.

Know Your Audience

The examples and suggestions shown above can go a long way toward getting an application in shape for dealing with date inputs from users in multiple cultures. There are some instances, however, where taking approaches like these would not be appropriate. Sometimes the provider or consumer of date values that pass through your application are not people, but rather other applications (or other portions of your own application). For example, if your site has a page that accepts a date as a query string parameter, you’ll probably want to format that date using invariant date format. Otherwise, the same URL could end up evaluating to a different page depending on the user that is viewing it. In addition, if your application exports data for consumption by other systems, it’s best to have an agreed upon format that all systems can use and that will not vary depending upon whether or not the users of the systems on either side prefer a month/day/year or day/month/year format. I’ll look more at some approaches for dealing with these situations in a future post.

If you take away one thing from this post, make it an understanding of the importance of knowing where the dates that pass through your system come from and are going to. You will likely want to vary your parsing and formatting approach depending on your audience.

• Share This Post:
• 
• 
• 

Posted On Monday, January 03, 2011 7:39 PM | Feedback (0) |