Doug.Instance

Improving the world one post at a time

  Home  |   Contact  |   Syndication    |   Login
  27 Posts | 0 Stories | 47 Comments | 0 Trackbacks

News

Tag Cloud


Archives

Post Categories

My Sites

Monday, November 10, 2014 #

First off, please see my internet PSA at the bottom of this post on the difference between “yea”, “yeah”, and “yay”.  Now that that is out of the way, I want to talk about settings.  Almost every settings example provided by Microsoft will look something like this which happens to be from the Azure queue documentation:

<configuration>
    <appSettings>
        <add key="StorageConnectionString" value="DefaultEndpointsProtocol=https;AccountName=account-name;AccountKey=account-key" />
    </appSettings>
</configuration>

 

I have seen this type of thing so many times that it caused me to question my own sanity.  Maybe there is a good reason for this.  I have a theory – it is easier to have a completely working demo when all of the configuration can be performed in your config file and code.  Using strongly-typed settings has multiple advantages but it takes a little more instruction to implement so include that little bit of how-to in every demo is unnecessary.  I have no idea why Microsoft uses appSettings for other common code, but let’s just say I hope one day this type of thing disappears:

<appSettings>
  <add key="webpages:Version" value="3.0.0.0" />
  <add key="webpages:Enabled" value="false" />
  <add key="ClientValidationEnabled" value="false" />
  <add key="UnobtrusiveJavaScriptEnabled" value="false" />
</appSettings>

Why Not appSettings?

The biggest difference between appSettings and applicationSettings is that applicationSettings are “strongly-typed”.  The best way to illustrate this is by showing how “appSettings” are NOT strongly typed.  As you see in the example above, appSettings can be added with an “add” XML element with two attributes: “key” and “value”.  Both “key” and “value” are treated as strings.  Therefore the setting above can be retrieved with the following code:

CloudConfigurationManager.GetSetting("StorageConnectionString")

 

This code passes the key in as a string and returns the value as a string (“DefaultEndpointsProtocol=https;AccountName=account-name;AccountKey=account-key”).  While that is OK if you are dealing with strings, it introduces some challenges if your code gets complex or if you use open source projects or other third-party code.  The problem arises because every value of “key” in appSettings must be unique.  That means you have to plan ahead and make sure your keys are unique and hope the third-party code you use does the same.

Why applicationSettings?

Now let’s add the same key using applicationSettings.  The first thing you will need to do is add a settings file to your project.  You MUST add this settings file to the project that will read the settings.  In the example below, I am adding this setting to an Azure WorkerRole project called “MyWorkerRole”, but this could be any project (web application, class library, etc.).  First access the project properties by either right-clicking on the project and select “Properties” or double-clicking “Properties” in inside the project in Solution Explorer.  Then select the Settings tab and click the link to add a settings file shown below.

image

Then add the setting.  Note that the default scope is “User”.  For client applications, this allows you to have settings that can be configured for each individual user.  For web applications and other server-side uses, you will need to change the scope to “Application”.  Note that the default variable type is “string” but you can select from common .Net types and also types defined in code.  Set the “Name” value to the desired name and enter the value in the “Value” property.

image

Now to retrieve this setting, we can just use the following property automatically added to a default instance of your strongly-typed settings:

    Properties.Settings.Default.StorageConnectionString

One of the benefits of this syntax is that the settings are compiled into your project.  This means you get intellisense on the settings properties (i.e. “StorageConnectionString”) so you don’t have to rely on string constants (AKA “magic strings”) to retrieve your settings.  Also, the values are strongly-typed.  If you select “bool” or “int” as the Type in your settings file, that property will evaluate as that type in your code.  No parsing or type conversion required!

One Last Thing!

Since we are using strongly-typed settings that must be stored in the project where the settings are created, the most common mistake you will make is forgetting to put the settings in the project where they are used.  Luckily this is easy since Visual Studio puts the settings in the app.config (or web.config) of the project where you created them.  Simply copy and paste the values into the config file of your main project (i.e. web.config in your web application).  Below is a sample config file.

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
    <configSections>
        <sectionGroup name="applicationSettings" type="System.Configuration.ApplicationSettingsGroup, System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" >
            <section name="MyWorkerRole.Properties.Settings" type="System.Configuration.ClientSettingsSection, System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" requirePermission="false" />
        </sectionGroup>
    </configSections>
    <applicationSettings>
        <MyWorkerRole.Properties.Settings>
            <setting name="StorageConnectionString" serializeAs="String">
                <value>DefaultEndpointsProtocol=https;AccountName=account-name;AccountKey=account-key</value>
            </setting>
        </MyWorkerRole.Properties.Settings>
    </applicationSettings>
</configuration>

In the “configSections” element, we define applicationSettings section and the specific settings section “MyWorkerRole.Properties.Settings”.  Note that this section name uses the namespace of the project so as long as your namespace is unique, your settings will be unique.  The best practice is to start your namespace with your company name or other unique identifier so a better namespace (and therefore project name) would be something like “MyCompany.MyProject.WorkerRole”.  Note that inside the “applicationSettings” section, there is a strongly-typed section with the same name as the sectionGroup defined above (“MyWorkerRole.Properties.Settings”).  This is where you can change your values.  You can also use standard xdt transformations to replace values based on build configurations.  For example, this is how you could do this in something like web.Release.config:

<applicationSettings>
    <MyWorkerRole.Properties.Settings>
        <setting name="StorageConnectionString" serializeAs="String" xdt:Locator="Match(name)">
            <value xdt:Transform="Replace">New value goes here</value>
        </setting>
    </MyWorkerRole.Properties.Settings>
</applicationSettings>

There is a potential benefit of appSettings in that various web-based GUIs allow you to modify values in appSettings. Since strongly-typed applicationSettings sections are dependent on the code, you usually have to modify the text or use this xdt transformation during publication to make changes on your server.

Internet Public Service Announcement

If you want to affirm something, then “yea” is probably your word.  This is pronounced “yay” and means “yes I approve” as in voting “yea” or “nay”.  Most likely, when you say “yay” you really mean “yea”.  If you say “yeah”, that is slang for “yes” and pronounced with a short “a” as in “as”.  If you use “yeah”, it probably isn’t inappropriate but it probably isn’t what you mean to say.  If you want to exclaim excitement, then you probably want to use “yay”.  Honestly I wasn’t sure that was a “real word” until I did some research.  Basically it is the exclamatory version of “yea”.  So much like all blimps are dirigibles but not all dirigibles are blimps, “yay” always means “yea” but “yea” does not always mean “yay”.  “Yeah” means “yes” and “yea” means “yes” but not always in the same way.  “Yeah” can mean “yay” but is never pronounced “yay”.


Wednesday, October 8, 2014 #

In my previous post, I outlined a general structure for setting up an application using node.js and other open tools in Visual Studio Express.  Now I would like to expand this to show how I would host the angular-seed starter application for AngularJS.  If you aren’t familiar with AngularJS or the seed app, you might want to take a look before continuing because I am going to assume you are already familiar with both.

First, I used the same File->New Project setup in my previous post creating a new solution, empty web application with MVC and Web API, and unit tests.  Then I set my web application as my StartUp project.  The resulting solution explorer looks like this:

image

In my previous post, I added a blank web site.  In this case, we want to use the angular-seed project so I extracted its contents to my solution folder:

image

Note that I don’t plan on contributing to the angular-seed project.  I am simply going to use it for a new application hosted in an ASP.Net MVC application so I downloaded the source as a zip rather than cloning the git repository.  If this were a project I planned on contributing to, I would clone the repository and leave the file structure unchanged.  Since I am building a new application, I would like my file structure to be “clean” so I am going to move all of these files into my web application project folder and add them to my project.  Then add the “app” and “e2e-tests” folder as existing web sites.  The resulting solution looks like this:

image

Before I start building the seed app in VS, lets install all of the components and take a look under the hood.  Assuming you have node.js installed, all  you have to do is navigate to the web application folder in your git bash and type “npm install”.  Note that this install uses bower to install components so you will need to have git installed so bower can retrieve the components.  Once the install completes, you can actually just run the angular app as-is from your “app” web site.  Just select the “Index.html” file, right click, and select “View in Browser”.  You may also want to refresh your app web site (right-click->Refresh Folder) so you can see the files added by the npm install.

image

Now that we have the app working the “angular way” (for the most part), let’s take a closer look at the CSS and JavaScript references in index.html.  First let’s look at the top of the file.

<link rel="stylesheet" href="bower_components/html5-boilerplate/css/normalize.css">
<link rel="stylesheet" href="bower_components/html5-boilerplate/css/main.css">
<link rel="stylesheet" href="app.css"/>
<script src="bower_components/html5-boilerplate/js/vendor/modernizr-2.6.2.min.js"></script>

The first 2 CSS files are part of the html5-boilerplate project.  The normalize.css file helps resolve differences between browsers and main.css provides some html5 best practices.  The app.css is the CSS for your aplication.  The last file, modernizr-2.6.2.min.js adds support for some html5 features to older browsers.

Now let’s look at the bottom of the file.

<!-- In production use:
<script src="//ajax.googleapis.com/ajax/libs/angularjs/x.x.x/angular.min.js"></script>
-->
<script src="bower_components/angular/angular.js"></script>
<script src="bower_components/angular-route/angular-route.js"></script>
<script src="app.js"></script>
<script src="view1/view1.js"></script>
<script src="view2/view2.js"></script>
<script src="components/version/version.js"></script>
<script src="components/version/version-directive.js"></script>
<script src="components/version/interpolate-filter.js"></script>

The commented out script and first 2 scripts are angular.  In this case we are only including the angular core and angular routing module.  The rest of the JavaScript files are sample files for your application.

Since part of my objective is to do this the “Visual Studio” way, now we will start using Visual Studio in a more standard fashion by installing some of these JavaScript files as NuGet packages.  Let’s make sure we have selected the web application project and install the angular core and route modules and modernizr:

image

PM> Install-Package AngularJS.Route

PM> Install-Package modernizr

We should now see these files in the “scripts” folder in our web project:

image

We also need our CSS so let’s create a “Content” folder and copy our “main.css” and “normalize.css” from bower_components/html5-boilerplate/css/ into it:

image

 

Now we can build a controller and view to duplicate the index.html content Visual Studio style.  Let’s start with an empty controller called HomeController:

image

image

Then add a view called “Index” to the Views\Home folder.  Add a standard razor MVC view (in this case I am using MVC 5) without a layout.  The reason we are not using a layout is because we are building an AngularJS single page app (SPA) and this is our single page.  Simply copy and paste the contents of the angular-seed index.html file in your app web site into the Index.cshtml view page we just created.  To get things working quickly, we will simply point all of our script and CSS references to the correct locations first at the top of Index.cshtml:

<link rel="stylesheet" href="~/Content/normalize.css"/>
<link rel="stylesheet" href="~/Content/main.css"/>
<link rel="stylesheet" href="~/app/app.css" />
<script src="~/Scripts/modernizr-2.8.3.js"></script>

Then at the bottom:

<script src="~/Scripts/angular.js"></script>
<script src="~/Scripts/angular-route.js"></script>
<script src="~/app/app.js"></script>
<script src="~/app/view1/view1.js"></script>
<script src="~/app/view2/view2.js"></script>
<script src="~/app/components/version/version.js"></script>
<script src="~/app/components/version/version-directive.js"></script>
<script src="~/app/components/version/interpolate-filter.js"></script>

Note that since Razor 2 and MVC 4, it is no longer required to use @Url.Content to map paths relative to the application root and we can simply use the tilde at the front of the URL.

The last thing we need to do to get the application running again is to change our routing to include the “app” folder in the path to our views in view1/view1.js and view2/view2.js:

$routeProvider.when('/view1', {
  templateUrl: 'app/view1/view1.html',
  controller: 'View1Ctrl'
});

$routeProvider.when('/view2', {
  templateUrl: 'app/view2/view2.html',
  controller: 'View2Ctrl'
});

Before running our app, let’s modify the application properties to change start action from “Current Page” to “Specific Page” to make sure the default URL is the root of the web site:

image

There is one other piece of MVC goodness we will add to our Index.cshtml view and it will go inside the <head> tag:

<base href="@Url.Content("~")" />

In this case we DO need to use Url.Content because unfortunately Razor does not resolve the URL unless the tilde is a prefix.  When it is the only content of the href attribute, it does not get resolved to the correct path.  Now if you host your application in a different path other than the root of your web site, angular will automatically find files based on relatives paths from this “base” URL.

Now we can run our MVC application with the same results:

image

One piece of this equation is still missing which is the publication of this web site will not include our angular code and angular view files under the current configuration.  This setup is only intended for local development.  In a future post, I will cover using open source tools to compile your angular app for deployment.


In the beginning, there was GET and POST. In those days, most of the World Wide Web was read-only and browsed with text-based browsers. Then came graphical browsers, CSS, e-commerce, SSL, streaming video, mobile devices, etc. Two things withstood the test of time: GET and POST (we’ll ignore the other request methods for poetic license). Web applications are, and have always been, “service-oriented”. But what exactly does that mean and how does it change your design?

What Do I Mean by SOA?

SOA stands for “service-oriented architecture”. It does not stand for “Enterprise Service Bus” (ESB), “Universal Description Discovery and Integration” (UDDI), “Web Service Definition Language” (WSDL), or “Simple Object Access Protocol” (SOAP). However, some people (mostly software or training vendors) will say you are “doing it wrong” if you say you are using SOA but aren’t using these things. I strongly disagree. If the acronym were DEXPSBA for “Discoverable Enterprise Cross-Platform SOAP Based Architecture”, then you would have an argument. As long as SOA stands for service-oriented, I take it to mean exactly that – the functionality of the application can be provided as a service. Keeping with the theme, note that I did not say, “must be provided as a service”. SOA just implies an orientation to services.

The next obvious question is, “what is a service?” That is a little more difficult to answer. Clearly Windows services and *nix daemons are “software services” which are services. Software as a Service (SaaS) and Infrastructure as a Service (IaaS) offerings claim to be “services”. The “service industry” typically refers to restaurant and bar workers. I’m sure SOA does not necessarily have to be oriented at waiting tables. Traditionally, the word “service” in SOA refers to discrete units of functionality that can be “consumed” by other applications. I emphasized the word “can” because of the “O” in SOA standing for “oriented”.

Your Web Application is Already SOA

So now let’s put the pieces together and take a look at a typical web application. Typically, a web application runs through an application tier (ASP.Net, JSP, PHP, etc.) that sits on top of the web server tier (IIS, Apache, etc.). That web application handles HTTP requests (typically GET or POST) and provides a response (typically HTML, XML, or JSON). Other than the self-described “agent” in the header of the HTTP request, your application does not know what browser (if any) is submitting that request. The reason I say “if any” is because any software capable of submitting a valid HTTP request will be processed by the server and application. This includes bots, viruses, malware, and of course browsers – all of which can either honestly represent themselves or report a different agent. By being agnostic to the “application” it is responding to, your web application is inherently “service-oriented” in the broadest term. In more traditional terms, your application uses a closed (or obfuscated) API to provide responses to a set of known requests. Your inbound message format is HTTP GET and POST and your outbound message format is typically HTML or JSON. Some of those requests may modify the state of data stored by the application. Any application capable of producing properly-formatted requests can consume your “services”.

What Should You Do Differently?

Since your application already is made up of a series of discrete services which are ambivalent to their consumers, you should validate each of the following for every transaction:

1. The person making the request is who they say they are.

2. The person making the request is authorized to perform the transaction.

3. The input provided for the request is valid.

4. The current state of the data allows for the transaction to occur.

5. The transaction has completed successfully.

We will look at each of these in more detail, but these steps don’t really do anything to make your application more service-oriented, they just deal with the fact that it already is. So before we go more in-depth, let’s add one more thing to the list:

6. The transaction can be consumed by other applications.

1. Verifying Identity

The first thing you need to do is secure your “service methods”. Each piece of code that handles incoming HTTP requests needs to assume the requestor is malicious. If you don’t care who processes a transaction gets the information returned in response, then you have nothing to worry about. For some transactions such as browsing products in an e-commerce application or possibly browsing data in an intranet application, you may not care who is performing the transaction or you may assume adequate security is provided by your infrastructure. However, for more sensitive transactions like accessing bank information, making credit card purchases, or accessing proprietary company data, you will want to be more careful. For public applications, this means you may want to use security certificates, an additional password challenge, multi-factor authentication, IP validation, CAPTCHA, and other techniques to reduce the risk of malicious impersonation. None of these methods are foolproof, but they are much better than assuming that anyone providing a valid session token is the person that token represents. The steps you take to secure the transaction should be proportional to the risk of having the wrong person perform the transaction.

2. Verifying Authorization

Just like there is no guarantee that the person making a request is who they say they are, there is also no guarantee that they are allowed to make the request. Caching user roles on the client, even if encrypted, may speed up transactions it really just means that whoever is making the request has a valid encrypted token and not necessarily that you gave it to them. If you’ve done step one above, you can be more certain that the person is who they say they are, but I recommend double-checking the database to see if they are really a system admin or not.

3. Verifying Data Inputs

If you want to aggravate a developer, find a URL that has a number as a parameter, pass in a letter and see if you get a nice error message saying that they are very sorry for your problem and that the error was recorded and the system administrator has been notified. Then write a script to hit that URL a few hundred thousand times. I’m not actually encouraging anyone to make DoS attacks but filling up a database transaction log with error messages is an easy way to do that. Filling up a developer’s e-mail isn’t very nice either. If real users are getting real errors, just filtering through the nonsense to find the real errors is a real nuisance. Just because your HTML and client-side script prevents users from submitting malicious HTML, SQL, or JavaScript in your request doesn’t mean it can’t be in the HTTP request. Of course the price of a product stored in a cookie in the users shopping cart can’t be assumed to be the correct price for a product. Just like number 1 above, based on the criticality of the transaction, you should validate data. Some of this can be done at the application tier or framework level and some needs to be done at the transaction level. The key is that you don’t assume that the inputs provided by a user are valid even if your client UI limits the possible inputs. At the end of the day you are just receiving a HTTP request.

4. Verifying the State of Data

Once again, just because previous transactions only allowed users to select “valid” data doesn’t mean that the current request is still valid. Users could have kept a page open for a long period of time allowing data to be changed by other users, users can resubmit submitted forms, and of course malicious users or code can submit what appears to be a valid request. The safest thing to do is to retrieve the latest data and validate that applicable rules are met.

5. Verify the Transaction Completed

Unless you are villain in a spy movie or a spoof of said spy movies, you shouldn’t just assume everything went according to plan and walk away. If you are using an API that returns a message, make sure you look at the message. If your data access or database tiers have error handling that could result in invalid data being ignored, make sure the transaction happened as planned. Don’t simply return the data submitted by the user and assume that it matches the record of authority.

6. Make it Consumable

If you have done 1-5, you have already made your application more secure and more reliable. You have also reduced the reliance of each individual HTTP request on “validation” done in your UI. Therefore each transaction can be considered a service since now any well-formatted request will result in the desired response. To make your “services” more consumable, you just need to be more sensitive to how a different application might want your service to respond. For example, you may respond to a request with an entire fresh page of HTML data when only one value is changed. That may be fine for your application but meaningless to most other applications. Consider providing another version of that transaction that either returns only the modified HTML or a JSON or JSONP result with the impacted data. By taking this approach from the get-go, your users may get faster response times, less “back button” angst, and an overall better experience.

In order to make “another version” of anything, you need an appropriate level of abstraction. This is a completely different architectural discussion and a topic for another day. At a minimum, you should separate your database transactions from your user interface. This allows for some degree of code reuse. The amount of abstraction you need is proportional to the amount of reuse. If you want to use multiple types of data stores and clients then you will need a high level of abstraction. If you just want multiple versions of the same transaction to produce multiple types of HTTP responses, you just need those methods to be extracted appropriately. Once you have achieved the proper level of abstraction, you may choose to create a SOAP or REST service to expose those abstracted methods. Most web application frameworks make this relatively easy assuming you already have the correct level of abstraction. Then you might want to publish the WSDL, update your UDDI, and fire up your USB. Then again you might not (http://www.zdnet.com/blog/gardner/dont-use-an-esb-unless-you-absolutely-positively-need-one-mule-cto-warns/3060).

Putting it All Together

There is nothing worse than getting good advice when it is too late to act on it. That is why the emphasis on this article is on architecture. Like most architectural decisions, these concepts are best put in action when the determination to enact them is made up front. Retroactively making your web application more service-oriented may not pay off dividends. However, when building a web application, consider the potential for reuse and weigh that vs. the impact of implementation. There is a finite amount of logic in every application. There are a finite number of transactions with a finite number of parameters. Your decision to expose those transactions as a service is the equivalent of putting one dish in the dishwasher instead of in the sink and if you are already writing the code the dishwasher is already open. Make your significant other/roommate (AKA other coders) happy by putting that fork in the dishwasher.


Sunday, October 5, 2014 #

Open source tools and frameworks are great, but when you use frameworks like AngularJS and LESS you can end up with a many files that get compiled down to just a few.  Keeping these files separate is good for development and debugging so it is important to keep all of the original source files under source control, but you don’t want them published to your production web site.

I am a fairly recent convert to AngularJS but when I took the plunge I was surprised that even with its massive popularity and obvious synergy with .Net, it was difficult to find guidance on exactly how best to setup a project in Visual Studio.  Once you drink the Angular Kool-Aid, you quickly learn that it tastes best with a side of node.js, Grunt/Gulp, etc.  I am also a big fan of Bootstrap so my CSS usually starts with the Bootstrap LESS source.  Using the full-blown version of Visual Studio, the Web Essentials extension can duplicate many of the things you would do using node.js such as compiling/minfying LESS and concatenating/minifying JavaScript.  Unfortunately Web Essentials isn’t available for the Express editions of Visual Studio so people using this great, free tool need a different alternative.  Also, if you want to contribute to one of the open source projects that use node.js, you will need to be able to use these tools but you may want to use them with VS.

For those of you who like spoilers, most of the “magic” that I have learned can be summed up with the following structure:

  • ProjectName [Solution]
    • ProjectName.Web [Web Application]
      • src [Web Site]
    • ProjectName.Web.Tests [Web Application Tests]

This structure makes it easy to keep all of your source organized with only “production” code in your web application.  I will answer “how” and “why” in the rest of this post. Like all good ideas (and yes, I recognize I just complimented myself), this setup was born out of necessity and presented itself after trying to do it the “normal” way.  So let’s walk through the setup and I’ll point out the key decision points.

Just like most of my favorite walkthroughs, this starts with File->New Project.

NewProject

There are some subtle nuances here.  Note that the “Create directory for solution” checkbox is checked.  This sort of comes down to preference but you do get some quirks in the file structure if this is turned off.  Namely your test project (which we will create in the next step) will be in a directory under your web project.  Another thing you will note is my naming convention.  I have been using this sort of structure for a long time and it serves me pretty well.  Many examples you see will have a simple name like “BobsMvcDemo” and therefore that is the namespace.  If you are reading this you probably use .Net and are used to the massive namespace structure.  Isn’t it logical to use a similar structure in your own code?  I’ve talked a little more about this philosophy in other posts but just understand that “OpenToolsSample” is my root namespace and I add “.Web” to the project name which also adds that to the namespace.  Note that the solution name is just the root namespace or “OpenToolsSample”.

Next let’s create our web application and test project:

WebProject

My preference is to create an empty with MVC, Web API and unit tests checked.  If this is your first MVC app, you might want to select “MVC” and check the Web API and unit tests block.  That will give you a project that will run right out of the box along with some good code and helpful NuGet packages to get you started.  However, I find it faster to add the things I want than to remove those that I don’t.  So if you know what your or doing (or if you just trust me), you can just keep following along to get a basic app up and running.

Once you hit OK you will see your web application and test projects.  Make sure your web application is set as the start-up project (right-click on the project in the Solution Explorer and select “Set as StartUp Project”).  Your solution should now look something like this:

image

If take a look at a typical open source project, you will usually see a package.json file in the root folder that can be used to configure node.js and modules.  You can install node.js in the root and all modules will be installed there as well.  The advantage of this is that changes to node.js and any modules you use don’t have to be incorporated until you want them to be.  You can have multiple projects all with different versions of node.js and modules without having to worry about conflicts.  Your configuration packages like Grunt (my preferred task runner), typically live in your root folder too.  Node and Grunt configuration is critical to building your application so it needs to be in source control (i.e. your Git repository).  It isn’t necessary to keep the modules themselves in source control since they can be installed from the configuration.  Therefore my recommendation is that you simply put your package.json and any other node configuration files (i.e. Gruntfile.js) in the root of your web application project.

So what about the rest of your code?  Regardless of the JavaScript framework you choose (Angular, Knockout, Ember, etc.), you will likely want to keep your source organized in multiple files.  The same is true for LESS.  Typically this folder is called “src” or “app”.  You could add this folder to your existing web application project (as I did) but then the default behavior would be to publish this source when you publish your web project.  There are a lot of ways to deal with this, but the one I settled on is to use a “web site” outside of your “web application” project so let’s add a new web site called “src” to our solution in the same location you would normally put this folder (under your web application project folder):

AddNewWebSite

image

Now our solution now looks like this (note I include package.json and Gruntfile.js for reference):

image

If I show all files from the web application project, you see that the “src” folder is actually under the physical path of the web application:

image

You can now add your solution and all associated projects to source control using your preferred source control provider (I use the free Team Foundation Server Express, but Git is also supported by VS Express).  You can put your node/Grunt/Gulp/etc. configuration in your web application as shown above with package.json and Gruntfile.js.  NuGet packages (AngularJS, Bootstrap, jQuery, etc.) would typically go in your web application project’s “Content” (for CSS) and “Scripts” (for JavaScript) folders so you can point your output from Grunt/Gulp there as well.  Then, following “normal” ASP.Net conventions, you can publish only your “production” code with your web application.  When you install node modules with “npm install”, the “node_modules” folder will be under your web application, not your “src” web site so that folder can be excluded from source control.

In my next post I will go into some specific examples using Grunt to concatenate and minify AngularJS source and to compile and minify LESS source.  I will also highlight some considerations for setting up AngularJS apps with MVC and Web API.


Tuesday, April 29, 2014 #

I have two quotes from old friends that have stuck in my head in certain situations.  The first is, "dancing is like standing still, only faster."  As a runner, I have adapted that to, "running is like falling down, only slower."  The second is, "when you hear hoofbeats, think of horses, not zebras."  As a programmer, I have found there is not much better advice than this second quote.

You would think that the more experience you have, the more likely you are to look for the obvious answer.  I've found that the opposite is true - while you are going to make "stupid" mistakes less frequently, the majority of your mistakes are still going to be "stupid".  Unfortunately with great power comes great ego, and the most difficult bugs are caused by the occurrence of something that was "never going to happen".  I happened to stumble through 3 of these self-induced hair-pulling-out exercises in a matter of days.  To make matters worse, I don't really code at work anymore.  This was a hobby project that I can only work on for an hour or two each night.

First was the infinite loop (and subsequent stack overflow) - one of the few errors that can't be captured with standard error logging techniques.  This problem only happened on the production server so instead of doing the obvious and duplicating the data down to dev so I could debug, I decided to add debug logging until I found the problem.  This was the first symptom of my bravado.  My application is a state machine "engine" that runs in a loop.  As things happen and the state changes, the loop eventually completes.  I narrowed down the problem to the state where the error occurred but had trouble getting closer than that.  I was convinced that some dark magic was at play and some data anomaly so I spent a few nights adding debugging, publishing, running, repeating.

Finally, I stopped looking for zebras.  Instead of continuing to try to zero in on the line of code through trial and error addition of debug logging, I put 3 lines of code in the most obvious places the problem could occur - the 3 while loops that could get called inside my main loop.  That immediately pointed me to the loop in question so then I threw an exception when a condition causing an infinite loop occurred.  It probably took me all of 5 minutes to determine the state that caused the problem.

So next I had to figure out how the code got into that state.  All of the information I needed to solve the rest of the problem, I saw within minutes - and completely ignored.  I saw that the state in question could only come from the database because the key data was never set anywhere by code.  Therefore, the code that allowed the infinite loop condition must have been in the code that loaded the data from the database.  However, rather than looking right where all indications pointed, I looked for zebras.  Since I knew how to find the problem, I could now duplicate it on my dev box and debug.  That meant I could spend time inspecting variables at breakpoints to see why the problem was happening.  After all, this infinite loop was caused by a recursive linked-list relationship - something that was "never going to happen".  There couldn't have been an obvious reason for something like this to happen so I had dig for a needle in the haystack instead of following the thread attached to the needle.  In spite of myself, I did eventually figure out that the problem could absolutely be solved when the data was loaded into the engine.  Technically this is after the data is extracted from the database, but the recursive situation doesn't matter until it gets to the engine so that's where I put the fix to truncate the recursive linked list since the duplicate reference shouldn't have been there anyway.

Fast forward a few days.  With my problem solved, I was able to run my engine for a few days and then started to look at the output.  Then I stumbled on a new error - "An error occurred while reading from the store provider's data reader. See the inner exception for details" and the inner exception was "Arithmetic overflow error for data type tinyint, value = 2288".  So, the expert programmer has managed to break the Entity Framework.  Of course I checked the "obvious" problem - that my model had the wrong data type.  Then I went off the deep end.  Unfortunately, there are people who have had legitimate problems with EF or linq-to-SQL mapping parameters incorrectly (2288 was a parameter, not a database value).  I converted my linq expression into a SQL command and then realized I was once again looking for zebras.  Instead of looking at the columns in my database that would map to tinyint, I assumed the problem was in the Entity Framework - code written by Microsoft and used by thousands if not millions of people - and not in MY code.  When I stepped through my code and looked at the parameters, I noticed there was a parameter that should't have even been there.  That was my problem - I passed the parameters into my method in the wrong order!  Instead of passing "int, null, int, true" I was passing "int, int, null, true".  The 2nd and 3rd parameters were both nullable ints so as far as the compiler was concerned this was all good.  Unfortunately it meant it passed my very large primary key into a parameter that was expecting a much smaller value - something that should have been very obvious from the error.

This post has been sitting in draft for so long that I have now had several more "events".  The most recent was while doing a demo on AngularJS directives with my team at work.  I was trying to illustrate scope isolation and no matter what I tried I couldn't figure out why the attribute from directive specified in my HTML wasn't making its way to my link function in the directive code.  When I turned off scope isolation and pointed at the parent scope, everything worked fine.  Even my more Angular-savvy team members were stumped.  Then came the "duh" moment - I actually had 2 directives nested inside each other.  I was looking at the code in the inner directive and passing data into the outer directive.  The HTML of the inner directive wasn't passing the value from the parent.  Yet another zebra chase.

So in conclusion, drop the attitude!  When you have problems, they are most likely the most basic and amateur issues.  Reign in those horses and let the zebras smack you in the head, don't go looking for them.

Sunday, March 10, 2013 #

In Part III, we started creating a very basic MVC application to host our store.  We also created a simple table to store products and methods to perform CRUD transactions and validate data.  Now we we introduce building product categories that can be related to products in a “many-to-many” relationship.  This means that multiple products can be assigned to a category and a product can be assigned to multiple categories. 

Creating the Tables

The category table is very simple and consists of a name and description.

/* ProductCategory.sql
* Copyright Streetlight Technologies L.L.C. All rights reserved.
*/
CREATE TABLE dbo.ProductCategory
(
    [ProductCategoryId] int identity(1, 1), -- Primary Key
    [Name] varchar(MAX), -- Prouct category name
    [Description] varchar(MAX), -- Product category description
    CONSTRAINT ProductCategory_PK PRIMARY KEY CLUSTERED
    (
        ProductCategoryId
    )
)
go

Next, we will create a table to relate categories to products.

/* ProductCategoryProduct.sql
* Copyright Streetlight Technologies L.L.C. All rights reserved.
*/
CREATE TABLE dbo.ProductCategoryProduct
(
    [ProductCategoryId] int not null, -- Primary Key
    [ProductId] int not null,
    CONSTRAINT ProductCategoryProduct_PK PRIMARY KEY CLUSTERED
    (
        ProductCategoryId,
        ProductId
    )
)
go

Then we create the foreign keys to link the three tables together.

ALTER TABLE dbo.ProductCategoryProduct
ADD CONSTRAINT ProductCategoryProduct_ProductCategory_FK FOREIGN KEY
(ProductCategoryId) REFERENCES ProductCategory(ProductCategoryId)
go

ALTER TABLE dbo.ProductCategoryProduct
ADD CONSTRAINT ProductCategoryProduct_Product_FK FOREIGN KEY
(ProductId) REFERENCES Product(ProductId)
go

After refreshing our entity model, it looks like this:

image

Note that the “ProductCategoryProduct” table does not appear in our model.  That is because there is no additional data other than foreign keys in the table so the Entity Framework considers the table trivial and handles it via the navigation properties in the Product and ProductCategory entities.  As we will see in the implementation of the database code, this isn’t ideal.  However, since it is the standard behavior for the Entity Framework and it should cause significant performance problems for a relatively low-traffic e-commerce site (we assume we are not hosting Amazon.com or eBay), we are going to work around this limitation.

Wiring Up the Data Access Layer

The first four methods to handle product categories are basically the same as the CRUD methods for products we saw in Part I.

/// <summary>
/// Returns a list of all categories.
/// </summary>
/// <returns>New instance of List&lt;ProductCategory&gt; containing data for all categories</returns>
public List<ProductCategory> ListAllCategories()
{
    return _dataManager
        .Context
        .ProductCategories
        .ToList()
        .Select(c => new ProductCategory
            {
                Id = Convert.ToString(c.ProductCategoryId),
                Name = c.Name,
                Description = c.Description
            })
        .ToList();
}

/// <summary>
/// Stores the category provided as a new record.
/// </summary>
/// <param name="category">Category to be saved</param>
public void CreateNewCategory(ProductCategory category)
{
    if (category == null)
    {
        throw new ArgumentNullException("category");
    }

    DataModels.ProductCategory categoryData = new DataModels.ProductCategory
    {
        Name = category.Name,
        Description = category.Description
    };

    _dataManager.Context.ProductCategories.Add(categoryData);
   
    _dataManager.Context.SaveChanges();

    category.Id = Convert.ToString(categoryData.ProductCategoryId);
}

/// <summary>
/// Gets the category with the specified ID.
/// </summary>
/// <param name="id">ID of category to return</param>
/// <returns>New instance of ProductCategory with data for provided ID.</returns>
public ProductCategory GetCategory(string id)
{
    int idValue = ConversionHelper.TryParsePositiveInt("id", id);

    DataModels.ProductCategory categoryData = _dataManager
        .Context
        .ProductCategories
        .FirstOrDefault(c => c.ProductCategoryId == idValue);

    if (categoryData == null)
    {
        return null;
    }

    return new ProductCategory { Id = id, Name = categoryData.Name, Description = categoryData.Description };
}

/// <summary>
/// Saves the provided category as an existing recod.
/// </summary>
/// <exception cref="System.InvalidOperationException">Thrown when no caetgory is found for the ID
/// provided as category.Id.</exception>
/// <param name="category">Category to save</param>
public void SaveCategory(ProductCategory category)
{
    if (category == null)
    {
        throw new ArgumentNullException("category");
    }

    int id = ConversionHelper.TryParsePositiveInt("category.Id", category.Id);

    DataModels.ProductCategory categoryData = _dataManager.Context.ProductCategories.FirstOrDefault(c => c.ProductCategoryId == id);

    if (category == null)
    {
        throw new InvalidOperationException(string.Format("Category not found for id {0}.", id));
    }

    categoryData.Name = category.Name;
    category.Description = category.Description;
   
    _dataManager.Context.SaveChanges();
}

Since we need to relate products to categories, we need to be able to find out what categories are already assigned to a product, assign categories, and remove categories.  The validation in this case is a little more simple.  We are going to follow the “law of least astonishment” and simply ignore an attempt to create a duplicate assignment or delete an assignment which does not exist.

/// <summary>
/// Lists categories associated with the specified ID.
/// </summary>
/// <param name="id">ID of product to list categories for</param>
/// <returns>New instance of List&lt;ProductCategory&gt; containing categories associated with product.</returns>
public List<ProductCategory> ListCategoriesForProduct(string id)
{
    int idValue = ConversionHelper.TryParsePositiveInt("id", id);

    DataModels.Product productData = _dataManager
        .Context
        .Products
        .Include("ProductCategories")
        .FirstOrDefault(p => p.ProductId == idValue);

    if (productData == null)
    {
        throw new InvalidOperationException(string.Format("Product not found for id {0}.", idValue));
    }

    return productData
        .ProductCategories
        .Select(c => new ProductCategory { Id = Convert.ToString(c.ProductCategoryId), Name = c.Name, Description = c.Description })
        .ToList();
}

/// <summary>
/// Adds the product with the specified ID to the category with the specified ID.  If a relationship
/// already exists, nothing is done.
/// </summary>
/// <param name="productId">ID of product to add</param>
/// <param name="categoryId">ID of category to add to</param>
public void AddProductToCategory(string productId, string categoryId)
{
    int productIdValue = ConversionHelper.TryParsePositiveInt("productId", productId);

    int categoryIdValue = ConversionHelper.TryParsePositiveInt("categoryId", categoryId);

    DataModels.ProductCategory category = _dataManager
        .Context
        .ProductCategories
        .Include("Products")
        .FirstOrDefault(c => c.ProductCategoryId == categoryIdValue);

    if (category == null)
    {
        throw new InvalidOperationException(string.Format("Category not found for id {0}.", categoryIdValue));
    }

    if (category.Products.Any(p => p.ProductId == productIdValue))
    {
        return;
    }

    DataModels.Product product = _dataManager
        .Context
        .Products
        .FirstOrDefault(p => p.ProductId == productIdValue);

    if (product == null)
    {
        throw new InvalidOperationException(string.Format("Product not found for id {0}.", productIdValue));
    }

    category.Products.Add(product);

    _dataManager.Context.SaveChanges();
}

/// <summary>
/// Removes the product with the specified ID from the category with the specified ID.  If no relationship
/// exists, nothing is done.
/// </summary>
/// <param name="productId">ID of product to remove</param>
/// <param name="categoryId">ID of category to remove from</param>
public void RemoveProductFromCategory(string productId, string categoryId)
{
    int productIdValue = ConversionHelper.TryParsePositiveInt("productId", productId);

    int categoryIdValue = ConversionHelper.TryParsePositiveInt("categoryId", categoryId);

    DataModels.ProductCategory category = _dataManager
        .Context
        .ProductCategories
        .Include("Products")
        .FirstOrDefault(c => c.ProductCategoryId == categoryIdValue);

    if (category == null)
    {
        throw new InvalidOperationException(string.Format("Category not found for id {0}.", categoryIdValue));
    }

    if (!category.Products.Any(p => p.ProductId == productIdValue))
    {
        return;
    }

    DataModels.Product product = _dataManager
        .Context
        .Products
        .FirstOrDefault(p => p.ProductId == productIdValue);

    if (product == null)
    {
        throw new InvalidOperationException(string.Format("Product not found for id {0}.", productIdValue));
    }

    category.Products.Remove(product);

    _dataManager.Context.SaveChanges();
}

 

Next, we create the ProductCategoriesController to do the CRUD transactions.

public class ProductCategoriesController : StoreController
{
    public ActionResult Index()
    {
        List<ProductCategory> model = TransactionManager.Products.ListAllCategories();

        return View("List", model);
    }

    public ActionResult List()
    {
        List<ProductCategory> model = TransactionManager.Products.ListAllCategories();

        return View(model);
    }

    public ActionResult Create()
    {
        ProductCategory model = new ProductCategory();

        return View(model);
    }

    [HttpPost]
    public ActionResult Create(ProductCategory model)
    {
        if (ModelState.IsValid)
        {
            TransactionManager.Products.CreateNewCategory(model);

            return RedirectToAction("Edit", new { id = model.Id });
        }

        return View(model);
    }

    public ActionResult Edit(string id)
    {
        ProductCategory model = TransactionManager.Products.GetCategory(id);

        return View(model);
    }

    [HttpPost]
    public ActionResult Edit(ProductCategory model)
    {
        if (ModelState.IsValid)
        {
            TransactionManager.Products.SaveCategory(model);
        }

        return View(model);
    }
}

Then, we add methods to the ProductsController to list categories and add and remove categories.

public ActionResult CategoryList(string id)
{
    ViewBag.Id = id;

    List<ProductCategory> model = TransactionManager.Products.ListCategoriesForProduct(id);

    return PartialView(model);
}

public ActionResult AddCategory(string id)
{
    ViewBag.Id = id;

    List<ProductCategory> model = TransactionManager.Products.ListAllCategories();

    return PartialView(model);
}

[HttpPost]
public ActionResult AddCategory(string id, string categoryId)
{
    TransactionManager.Products.AddProductToCategory(id, categoryId);

    return RedirectToAction("Edit", new { id = id });
}

public ActionResult RemoveCategory(string id, string categoryId)
{
    TransactionManager.Products.RemoveProductFromCategory(id, categoryId);

    return RedirectToAction("Edit", new { id = id });
}

The CategoryList action displays a partial view with a list of categories associated with a product and links to remove the category. The AddCategory action displays a partial view with a drop-down list to select a category and a button to add it to the product.  The resulting product view looks like this:

image

Download source code from CodePlex.


Wednesday, March 6, 2013 #

To keep things simple, I am going to retain the default security framework created by Visual Studio when I created the application.  However, I am going to change the database name to something more meaningful.  So in web.config, I am changing the connection string to this:

<add name="DefaultConnection" connectionString="Data Source=(LocalDb)\v11.0;Initial Catalog=StreetlightStoreSample;Integrated Security=SSPI;AttachDBFilename=|DataDirectory|\StreetlightStoreSample.mdf" providerName="System.Data.SqlClient" />

Next we’ll add the connection string from our database project found in Streetlight.Store.DataAccess\app.config:

<add name="StoreEntities" connectionString="metadata=res://*/DataModels.StoreDataModel.csdl|res://*/DataModels.StoreDataModel.ssdl|res://*/DataModels.StoreDataModel.msl;provider=System.Data.SqlClient;provider connection string=&quot;data source=.\SQLEXPRESS;initial catalog=SAMPLE_STORE;integrated security=True;MultipleActiveResultSets=True;App=EntityFramework&quot;" providerName="System.Data.EntityClient" />

Now let’s look at the home page which is the Views\Home\Index.cshtml view.  Most of the changes are cosmetic until we get to the lists at the bottom of the page.  Here we will link to some actions to be added to the MVC application:

<ol class="round">
    <li class="one">
        <h5>Store Management</h5>
        Manage your store by creating products and handling customer orders.
        @Html.ActionLink("Manage store…", "Index", "Admin")
    </li>

    <li class="two">
        <h5>Browse Products</h5>
        Customers hopefully want to buy your products so let's
        @Html.ActionLink("browse products…", "Index", "Products")
    </li>

    <li class="three">
        <h5>User Customization</h5>
        @if (Request.IsAuthenticated)
        {
            <text>Hey! You registered!  Now you can look at your past orders and
            manage your account information.</text>
        }
        else
        {
            <text>@Html.ActionLink("Register", "Register", "Account", routeValues: null, htmlAttributes: new { id = "registerLink" }) or
            @Html.ActionLink("Log in", "Login", "Account", routeValues: null, htmlAttributes: new { id = "loginLink" })
            to view past orders and manage your account information.</text>
        }
    </li>
</ol>

 

Next we will change the header menu found in \Views\Shared\_Layout.cshtml:

<nav>
    <ul id="menu">
        <li>@Html.ActionLink("Home", "Index", "Home")</li>
        <li>@Html.ActionLink("Admin", "Index", "Admin")</li>
        <li>@Html.ActionLink("Shop", "Index", "Store")</li>
    </ul>
</nav>

 

As a result, our new home page looks like this:

image

Note that this also required a change to Controllers\HomeController.cs controller class to change the message displayed by the Index action.

The Products Controller

Let’s create a ProductsController controller class to call the methods we created in Part II to manage products.  The first code we need to add to this class is the code to handle creation and disposal of the transaction manager:

public ITransactionManager TransactionManager { get; private set; }

public ProductsController()
{
    TransactionManager = new Streetlight.Store.DataAccess.DataManager();
}

protected override void Dispose(bool disposing)
{
    if (disposing && TransactionManager != null)
    {
        TransactionManager.Dispose();
    }

    base.Dispose(disposing);
}

 

Note that in this case we create a strongly-typed instance of the specific implementation of ITransactionManager (Streetlight.Store.DataAccess.DataManager) introduced in Part II.  In future posts, we will look at further abstracting the initialization of the TransactionManager property.

Next we will create the get and post actions to create a new product:

public ActionResult Create()
{
    Product model = new Product();

    return View(model);
}

[HttpPost]
public ActionResult Create(Product model)
{
    if (TransactionManager.Products.DuplicateItemNumberExists(string.Empty, model.ItemNumber))
    {
        ModelState.AddModelError("ItemNumber", "A product already exists with this item number.");
    }

    if (TransactionManager.Products.DuplicateGlobalIdManufacturerExists(string.Empty, model.GlobalId, model.Manufacturer))
    {
        ModelState.AddModelError("GlobalId", "A product already exists with this Global ID/Manufacturer combination.");
    }

    if (ModelState.IsValid)
    {
        TransactionManager.Products.CreateNew(model);

        return RedirectToAction("Edit", new { id = model.Id });
    }
    else
    {
        return View(model);
    }
}

 

To create the view to create a new product, we will use the default scaffolding in Visual Studio:

image

For now we will remove the status field and change the title so the final view looks like this in Views\Products\Create.cshtml:

@model Streetlight.Store.Contracts.Product

@{
    ViewBag.Title = "New Product";
}

<h2>New Product</h2>

@using (Html.BeginForm()) {
    @Html.ValidationSummary(true)

    <fieldset>
        <legend>Product</legend>

        <div class="editor-label">
            @Html.LabelFor(model => model.Name)
        </div>
        <div class="editor-field">
            @Html.EditorFor(model => model.Name)
            @Html.ValidationMessageFor(model => model.Name)
        </div>

        <div class="editor-label">
            @Html.LabelFor(model => model.Price)
        </div>
        <div class="editor-field">
            @Html.EditorFor(model => model.Price)
            @Html.ValidationMessageFor(model => model.Price)
        </div>

        <div class="editor-label">
            @Html.LabelFor(model => model.PackagingTypeCode)
        </div>
        <div class="editor-field">
            @Html.EditorFor(model => model.PackagingTypeCode)
            @Html.ValidationMessageFor(model => model.PackagingTypeCode)
        </div>

        <div class="editor-label">
            @Html.LabelFor(model => model.GlobalId)
        </div>
        <div class="editor-field">
            @Html.EditorFor(model => model.GlobalId)
            @Html.ValidationMessageFor(model => model.GlobalId)
        </div>

        <div class="editor-label">
            @Html.LabelFor(model => model.Manufacturer)
        </div>
        <div class="editor-field">
            @Html.EditorFor(model => model.Manufacturer)
            @Html.ValidationMessageFor(model => model.Manufacturer)
        </div>

        <div class="editor-label">
            @Html.LabelFor(model => model.PartNumber)
        </div>
        <div class="editor-field">
            @Html.EditorFor(model => model.PartNumber)
            @Html.ValidationMessageFor(model => model.PartNumber)
        </div>

        <div class="editor-label">
            @Html.LabelFor(model => model.Weight)
        </div>
        <div class="editor-field">
            @Html.EditorFor(model => model.Weight)
            @Html.ValidationMessageFor(model => model.Weight)
        </div>

        <div class="editor-label">
            @Html.LabelFor(model => model.Description)
        </div>
        <div class="editor-field">
            @Html.EditorFor(model => model.Description)
            @Html.ValidationMessageFor(model => model.Description)
        </div>

        <div class="editor-label">
            @Html.LabelFor(model => model.ItemNumber)
        </div>
        <div class="editor-field">
            @Html.EditorFor(model => model.ItemNumber)
            @Html.ValidationMessageFor(model => model.ItemNumber)
        </div>

        <div class="editor-label">
            @Html.LabelFor(model => model.Length)
        </div>
        <div class="editor-field">
            @Html.EditorFor(model => model.Length)
            @Html.ValidationMessageFor(model => model.Length)
        </div>

        <div class="editor-label">
            @Html.LabelFor(model => model.Width)
        </div>
        <div class="editor-field">
            @Html.EditorFor(model => model.Width)
            @Html.ValidationMessageFor(model => model.Width)
        </div>

        <div class="editor-label">
            @Html.LabelFor(model => model.Height)
        </div>
        <div class="editor-field">
            @Html.EditorFor(model => model.Height)
            @Html.ValidationMessageFor(model => model.Height)
        </div>

        <p>
            <input type="submit" value="Create" />
        </p>
    </fieldset>
}

<div>
    @Html.ActionLink("Back to List", "Index")
</div>

@section Scripts {
    @Scripts.Render("~/bundles/jqueryval")
}

 

We can then create the List and Edit actions and view using the default scaffolding as well with more minimal changes.

Download the full source code from CodePlex.


Tuesday, March 5, 2013 #

In Part I, I introduced the concept of the Streetlight Store and the architecture.  In this segment, we will walk through the creation of a data contract for products, interfaces for managing transactions, and a database-first Entity Framework implementation of the interfaces.

The initial plan was to distribute the store as a single library with the intent of using a SQL server database with a fixed schema.  However, just as flexibility is important in the front end, it is also important in the back end so I decided to abstract the back-end logic.  The namespaces listed in Part I (Streetlight.Store.Contracts and Streetlight.Store.DataAccess) have been separated into two assemblies.  The DataManager class described in Part I has been abstracted into the Streetlight.Store.Contracts.ITransactionManager interface and the other DataManagers have been similarly abstracted into IProductManager, IAccountManager, IOrderManager, and IShipmentManager.

The Product Data Contract

As I mentioned in Part I, the Streetlight Store is centered around POCO classes.  The first of these classes we are going to look at is the Product class.  This class encapsulates all of the pertinent information about a product.

namespace Streetlight.Store.Contracts
{
    using System; 

    public class Product
    { 
        public string Id { get; set; }
        public string Name { get; set; }
        public decimal? Price { get; set; }

        public string PriceText
        {
            get
            {
                return string.Format("{0:C}", Price);
            }
        }

        public string PackagingTypeCode { get; set; }
        public string GlobalId { get; set; }
        public string Manufacturer { get; set; }
        public string PartNumber { get; set; }
        public float? Weight { get; set; }
        public string Description { get; set; }
        public string Status { get; set; }
        public string ItemNumber { get; set; }
        public float? Length { get; set; }
        public float? Width { get; set; }
        public float? Height { get; set; }
    }
}

Database Schema

I’m sure some of you are asking, “Why database-first?  Why not use code-first or model-first?”  The short answer is I already have a database.  While I’m sure it is possible to use one of the other techniques, simply generating the model from the database was the easiest approach.

CREATE TABLE dbo.Product
(
    [ProductId] int identity(1, 1), -- Primary Key
    [Name] varchar(MAX), -- Prouct name
    [Price] decimal(18, 2), -- Price of product
    [PackagingTypeCode] varchar(50), -- Packaging type for shipping
    [GlobalId] varchar(MAX), -- Global Trade Item Number (GTID)
    [Manufacturer] varchar(MAX), -- Manufacturer's name
    [PartNumber] varchar(MAX), -- Manufacturer's part number
    [Weight] real, -- Weight of product
    [Description] varchar(MAX), -- Product description
    [StatusId] tinyint, -- Status ID
    [ItemNumber] varchar(MAX), -- Internal unique identifier
    [Length] real, -- Length of product
    [Width] real, -- Width of product
    [Height] real, -- Height of product
    CONSTRAINT Product_PK PRIMARY KEY CLUSTERED
    (
        ProductId
    )
)
go

 

Notice that not everything is an exact match between the database schema and the object model.  Strings are used instead of numeric types in the object model to allow flexibility in the back end.  Also, the naming convention of the object model is not identical to the database schema.  For example, the ProductId database field maps to the Id property of the object model.  A string is used in the object model instead of an integer so that GUIDs and other non-integer and non-numeric primary keys are supported.

Interfaces or “Operation Contracts”

Now that we have our database and object model, we need to be able to do some basic CRUD operations.  Following the paradigm established in Part I, the ITransactionManager interface contains properties for the other manager interfaces.  The other interfaces define the operations that can be performed following the “operation contract” paradigm from WCF.  For now we will focus just on the IProductManager and ITransactionManager.  These interfaces loosely follow unit of work pattern (ITransactionManager) and repository pattern (IProductManager).

namespace Streetlight.Store.Contracts
{
    using System;

    public interface ITransactionManager : IDisposable
    {
        IProductManager Products { get; }
    }
}

 

In the following code for IProductManager, four methods perform CRUD transactions while the remaining two (DuplicateItemNumberExists, DuplicateGlobalIdManufacturerExists) can be used for validation.

namespace Streetlight.Store.Contracts
{
    using System;
    using System.Collections.Generic;

    /// <summary>
    /// Provides an interface for managing product data.
    /// </summary>

    public interface IProductManager
    {
        /// <summary>
        /// Returns a list of all products.
        /// </summary>
        /// <returns>New instance of List<Product> containing data for all products</returns>

        List<Product> ListAll();

        /// <summary>
        /// Stores the provided Product as a new record.
        /// </summary>
        /// <param name="product">Product to be saved</param>

        void CreateNew(Product product);

        /// <summary>
        /// Checks to see if a product exists with the specified item number and different
        /// ID.  This function can be used to ensure exclusivity of item numbers.  Returns true
        /// if a product exists with the specified item number and different ID, otherwise
        /// returns false.
        /// </summary>
        /// <param name="id">ID of item to validate item number for.  If parameter is null,
        /// empty, or whitespace, all existing items will be searched.</param>
        /// <param name="itemNumber">Item number to check for duplicates</param>
        /// <returns>True if a product exists with the specified product number and different ID,
        /// otherwise false.</returns>

        bool DuplicateItemNumberExists(string id, string itemNumber);

        /// <summary>
        /// Checks to see if a product exists with the specified global ID and manufacturer and
        /// different ID.  This function can be used to ensure exclusivity of Global ID/
        /// Manufacturer combinations.  Returns true if a product exists with the specified
        /// global ID and manufacturer and different ID, otherwise returns false.
        /// </summary>
        /// <param name="id">ID of item to validate item number for.  If parameter is null,
        /// empty, or whitespace, all existing items will be searched.</param>
        /// <param name="globalId">Global ID to find duplicates for</param>
        /// <param name="manufacturer">Manufacturer to find duplicates for</param>
        /// <returns>True if a product exists with the specified
        /// global ID and manufacturer and different ID, otherwise false.</returns>

        bool DuplicateGlobalIdManufacturerExists(string id, string globalId, string manufacturer);

        /// <summary>
        /// Gets the product with the specified id.
        /// </summary>
        /// <param name="id">ID of product to retrieve</param>
        /// <returns>New instance of Product with data for provided ID.</returns>

        Product GetProduct(string id);

        /// <summary>
        /// Saves the provided Product as an existing record.
        /// </summary>
        /// <exception cref="System.InvalidOperationException">Thrown when no product is found for the ID
        /// provided as product.Id.</exception>
        /// <param name="product">Product to save</param>

        void Save(Product product);
    }
}

Putting the Pieces Together

Now that we have our object model, our database schema, and our operation contract, let’s actually implement the contract to interact with the database.  Most of this code is pretty straight-forward as defined in the comments with the exception of the last four methods: CreateContract, CopyToContract, CreateEntity, and CopyToEntity.  These methods deal with the conversions between the model class created by the Entity Framework and the data contract class.  While it is possible to map POCO classes to entity models, it is not always best practice map your POCO classes exactly to your database.  For example, in this implementation we use a ProductStatuses enum top populate the Product.Status property which is stored as a string.  The translation to and from the enum values is performed by the CopyToContract and CopyToEntity methods.  While this may seem cumbersome on the surface, it gives you very granular control over mapping the object model to the database.

//-----------------------------------------------------------------------
// <copyright file="ProductDataManager.cs" company="Streetlight Technologies L.L.C.">
//    Copyright Streetlight Technologies L.L.C. All rights reserved.
// </copyright>
//-----------------------------------------------------------------------

namespace Streetlight.Store.DataAccess
{
    using System;
    using System.Collections.Generic;
    using System.Linq;
    using Streetlight.Store.Contracts;

    /// <summary>
    /// Provides an instance of IProductManager using the Entity Framework to manage Product data.
    /// </summary>

    public class ProductDataManager : IProductManager
    {
        /// <summary>
        /// Instance of DataManager
        /// </summary>

        private DataManager _dataManager;

        /// <summary>
        /// Initializes a new instance of the <see cref="ProductDataManager"/> class.
        /// </summary>
        /// <param name="dataManager">DataManager instance containing this ProductDataManager instance.</param>
        public ProductDataManager(DataManager dataManager)
        {
            _dataManager = dataManager;
        }

        /// <summary>
        /// Returns a list of all products.
        /// </summary>
        /// <returns>New instance of List<Product> containing data for all products</returns>
        public List<Product> ListAll()
        {
            return _dataManager
                .Context
                .Products
                .ToList()
                .Select(p => CreateContract(p))
                .ToList();
        }

        /// <summary>
        /// Stores the provided Product as a new record.
        /// </summary>
        /// <param name="product">Product to be saved</param>

        public void CreateNew(Product product)
        {
            DataModels.Product productData = CreateEntity(product);

            _dataManager.Context.Products.Add(productData);

            _dataManager.Context.SaveChanges();

            product.Id = Convert.ToString(productData.ProductId);
        }

        /// <summary>
        /// Checks to see if a product exists with the specified item number and different
        /// ID.  This function can be used to ensure exclusivity of item numbers.  Returns true
        /// if a product exists with the specified item number and different ID, otherwise
        /// returns false.
        /// </summary>
        /// <param name="id">ID of item to validate item number for.  If parameter is null,
        /// empty, or whitespace, all existing items will be searched.</param>
        /// <param name="itemNumber">Item number to check for duplicates</param>
        /// <returns>True if a product exists with the specified product number and different ID,
        /// otherwise false.</returns>
        public bool DuplicateItemNumberExists(string id, string itemNumber)
        {
            IQueryable<DataModels.Product> query = _dataManager.Context.Products.Where(p => p.ItemNumber == itemNumber);

            if (!string.IsNullOrWhiteSpace(id))
            {
                int idValue = ConversionHelper.TryParsePositiveInt("id", id);

                query = query.Where(p => p.ProductId != idValue);
            }

            return query.Any();
        }

        /// <summary>
        /// Checks to see if a product exists with the specified global ID and manufacturer and
        /// different ID.  This function can be used to ensure exclusivity of Global ID/
        /// Manufacturer combinations.  Returns true if a product exists with the specified
        /// global ID and manufacturer and different ID, otherwise returns false.
        /// </summary>
        /// <param name="id">ID of item to validate item number for.  If parameter is null,
        /// empty, or whitespace, all existing items will be searched.</param>
        /// <param name="globalId">Global ID to find duplicates for</param>
        /// <param name="manufacturer">Manufacturer to find duplicates for</param>
        /// <returns>True if a product exists with the specified
        /// global ID and manufacturer and different ID, otherwise false.</returns>

        public bool DuplicateGlobalIdManufacturerExists(string id, string globalId, string manufacturer)
        {
            IQueryable<DataModels.Product> query = _dataManager.Context.Products.Where(p => p.GlobalId == globalId && p.Manufacturer == manufacturer);

            if (!string.IsNullOrWhiteSpace(id))
            {
                int idValue = ConversionHelper.TryParsePositiveInt("id", id);

                query = query.Where(p => p.ProductId != idValue);
            }

            return query.Any();
        }

        /// <summary>
        /// Gets the product with the specified id.
        /// </summary>
        /// <param name="id">ID of product to retrieve</param>
        /// <returns>New instance of Product with data for provided ID.</returns>
        public Product GetProduct(string id)
        {
            int idValue;

            if (!int.TryParse(id, out idValue))
            {
                throw new InvalidOperationException("Parameer \"id\" must evaluate to an integer.");
            }

            DataAccess.DataModels.Product productData = _dataManager.Context.Products.FirstOrDefault(p => p.ProductId == idValue);

            if (productData == null)
            {
                return null;
            }
            else
            {
                return CreateContract(productData);
            }
        }

        /// <summary>
        /// Saves the provided Product as an existing record.
        /// </summary>
        /// <exception cref="System.InvalidOperationException">Thrown when no product is found for the ID
        /// provided as product.Id.</exception>
        /// <param name="product">Product to save</param>

        public void Save(Product product)
        {
            if (product == null)
            {
                throw new ArgumentNullException("product");
            }

            int id;

            if (!int.TryParse(product.Id, out id))
            {
                throw new InvalidOperationException("Parameer \"id\" must evaluate to an integer.");
            }

            DataModels.Product productData = _dataManager.Context.Products.FirstOrDefault(p => p.ProductId == id);

            if (productData == null)
            {
                throw new InvalidOperationException(string.Format("Product not found for id {0}.", id));
            }

            CopyToEntity(product, productData);

            _dataManager.Context.SaveChanges();

            CopyToContract(productData, product);
        }

        /// <summary>
        /// Creates a new instance of Contracts.Product and copies values from specified DataModels.Product instance.
        /// </summary>
        /// <param name="productData">DataModels.Product containing data</param>
        /// <returns>New instance of Contracts.Product</returns>

        private static Product CreateContract(DataAccess.DataModels.Product productData)
        {
            Product product = new Product();

            CopyToContract(productData, product);

            return product;
        }

        /// <summary>
        /// Copies values from DataModels.Product to Contracts.Product.
        /// </summary>
        /// <param name="productData">DataModels.Product containing values to copy</param>
        /// <param name="product">Contracts.Product to copy to</param>
        private static void CopyToContract(DataAccess.DataModels.Product productData, Product product)
        {
            product.Id = Convert.ToString(productData.ProductId);
            product.ItemNumber = productData.ItemNumber;
            product.GlobalId = productData.GlobalId;
            product.Name = productData.Name;
            product.Manufacturer = productData.Manufacturer;
            product.PartNumber = productData.PartNumber;
            product.Description = productData.Description;
            product.Price = productData.Price;
            product.Length = productData.Length;
            product.Width = productData.Width;
            product.Height = productData.Height;
            product.Weight = productData.Weight;
            product.PackagingTypeCode = productData.PackagingTypeCode;

            if (productData.StatusId.HasValue)
            {
                product.Status = Convert.ToString((ProductStatuses)productData.StatusId);
            }
        }

        /// <summary>
        /// Creates a new instance of DataModels.Product and copies values from specified Contracts.Product instance.
        /// </summary>
        /// <param name="product">Contracts.Product containing data</param>
        /// <returns>New instance of DataModels.Product</returns>

        private static DataAccess.DataModels.Product CreateEntity(Product product)
        {
            DataModels.Product productData = new DataModels.Product();

            CopyToEntity(product, productData);

            return productData;
        }

        /// <summary>
        /// Copies values from Contracts.Product to DataModels.Product.
        /// </summary>
        /// <param name="product">Contracts.Product containing values to copy</param>
        /// <param name="productData">DataModels.Product to copy to</param>
        private static void CopyToEntity(Product product, DataAccess.DataModels.Product productData)
        {
            productData.ItemNumber = product.ItemNumber;
            productData.GlobalId = product.GlobalId;
            productData.Name = product.Name;
            productData.Manufacturer = product.Manufacturer;
            productData.PartNumber = product.PartNumber;
            productData.Description = product.Description;
            productData.Price = product.Price;
            productData.Length = product.Length;
            productData.Width = product.Width;
            productData.Height = product.Height;
            productData.Weight = product.Weight;
            productData.PackagingTypeCode = product.PackagingTypeCode;
           
            if (!string.IsNullOrEmpty(product.Status))
            {
                productData.StatusId = (byte)Enum.Parse(typeof(ProductStatuses), product.Status);
            }
        }
    }
}


 

Download the source code from CodePlex.


Wednesday, December 26, 2012 #

After looking at FluentValidation, I decided that most of what I needed was a way to do server-side validation of more complicated rules.  While this is something FluentValidation does, it seemed like more than I needed so I set out to create a simple extension method to do what I needed.  While it is still not necessarily perfect, it is simple and works.

   1:          public static void Validate<T, TResult>(this Controller controller, T model, Expression<Func<T, TResult>> expression, Func<TResult, bool> predicate, string message)
   2:          {
   3:              MemberExpression memberExpression = expression.Body as MemberExpression;
   4:   
   5:              if (memberExpression == null)
   6:              {
   7:                  throw new ArgumentException("expression must be a property.");
   8:              }
   9:   
  10:              PropertyInfo propertyInfo = memberExpression.Member as PropertyInfo;
  11:   
  12:              if (propertyInfo == null)
  13:              {
  14:                  throw new ArgumentException("expression must be a property.");
  15:              }
  16:   
  17:              TResult result = (TResult)propertyInfo.GetValue(model, new object[0]);
  18:   
  19:              if (!predicate(result))
  20:              {
  21:                  string key = propertyInfo.Name;
  22:               
  23:                  controller.ModelState.AddModelError(key, message);
  24:              }
  25:          }

Here is a sample implementation.  Obviously this is a trivial case but any condition can be used and will add the error if the expression evaluates to false.

 

   1:              this.Validate(
   2:                  model, // Model
   3:                  m => m.Name, // Parameter to use for key
   4:                  n => !string.IsNullOrEmpty(n), // Expression to add error if false
   5:                  "Name cannot be blank."); // Message to add if expression returns false


Friday, April 20, 2012 #

SharePoint is very handy as a repository for data.  There are things (particularly in the intranet world) which really don't need a dedicated database or application and SharePoint is a great enabler.  However, if you want to build a lightweight, standards-friendly web page using SharePoint, you have a challenge ahead of you.

Here is the problem I was trying to solve:  I have a SharePoint site that serves as the home page for an organization.  I needed to be able to pull in various content from other sites (on other servers) and present it in a nice, clean layout.  In order to keep the maintenance of this page as simple as possible, I wanted to use JavaScript/jQuery to fetch data and then use as much standard HTML/CSS for the page itself as possible.

Luckily, SharePoint has several client APIs.  There are some fairly easy ways you can get data from SharePoint lists (the fundamental data structure used by SharePoint).  I was able to find a workable solution rather quickly to pull list data in using the ASP.Net web services via jQuery.  This worked just fine until I found some content on a different server and ran into the cross-site scripting limitations of jQuery ajax.

In order to continue, I knew I would have to use JSONP.  The easiest way I could think of to do this would be to write some code to act as an intermediary between the JavaScript client and the SharePoint servers.  This intermediary utility was broken up into two methods to perform these 2 functions:

  1. Retrieve a SharePoint list as an ADO.Net DataSet.
  2. Retrieve a SharePoint list as JSONP.

The first step was to create an ASP.Net MVC project and add references to Microsoft.SharePoint.Client and Microsoft.SharePoint.Client.Runtime. 

Next I created the method to get a DataSet.  I understand DataSets aren't the "best" vehicle for transferring data, but they are ubiquitous so I felt this was the best short-term method to ensure potential reuse.  That method looks like this:

   1:          public static DataSet GetListAsDataSet(string serverUrl, string listName, string query)
   2:          {
   3:              ClientContext context = new ClientContext(serverUrl);
   4:   
   5:              List list = context.Web.Lists.GetByTitle(listName);
   6:   
   7:              if (list != null)
   8:              {
   9:                  context.Load(list);
  10:   
  11:                  FieldCollection fields = list.Fields;
  12:   
  13:                  context.Load(fields);
  14:                  context.ExecuteQuery();
  15:   
  16:                  DataSet ds = new DataSet();
  17:                  DataTable table = new DataTable();
  18:                  ds.Tables.Add(table);
  19:   
  20:                  foreach (Field field in fields)
  21:                  {
  22:                      if (!table.Columns.Cast<DataColumn>().Any(c => c.ColumnName == field.Title))
  23:                      {
  24:                          table.Columns.Add(field.Title, typeof(object));
  25:                          string temp = field.InternalName;
  26:                      }
  27:                  }
  28:   
  29:                  CamlQuery camlQuery = new CamlQuery();
  30:   
  31:                  if (!string.IsNullOrEmpty(query))
  32:                  {
  33:                      camlQuery.ViewXml = query;
  34:                  }
  35:   
  36:                  ListItemCollection items = list.GetItems(camlQuery);
  37:   
  38:                  context.Load(items);
  39:                  context.ExecuteQuery();
  40:   
  41:                  foreach (ListItem item in items)
  42:                  {
  43:                      DataRow row = table.NewRow();
  44:                      table.Rows.Add(row);
  45:   
  46:                      foreach (Field field in fields)
  47:                      {
  48:                          string key = field.Title;
  49:   
  50:                          if (item.FieldValues.ContainsKey(field.InternalName))
  51:                          {
  52:                              if (item[field.InternalName] == null)
  53:                              {
  54:                                  row[key] = DBNull.Value;
  55:                              }
  56:                              else
  57:                              {
  58:                                  row[key] = item[field.InternalName];
  59:                              }
  60:                          }
  61:                      }
  62:                  }
  63:   
  64:                  return ds;
  65:              }
  66:   
  67:              return null;
  68:          }

I then created an action method in my MVC controller to get the DataSet created by the list and convert it to an object with 2 properties.  The "fields" property contains an array of field names.  The "rowData" property contains an array of arrays representing rows and columns of data.  That method looks like this:

   1:          public ActionResult GetListAsJsonp(string server, string listName, int? rowLimit, string callback)
   2:          {
   3:              string query = null;
   4:   
   5:              if (rowLimit.HasValue)
   6:              {
   7:                  query = string.Format("<View><RowLimit>{0}</RowLimit></View>", rowLimit);
   8:              }
   9:   
  10:              DataSet ds = Quality.Common.SharePoint.ListHelper.GetListAsDataSet(server, listName, query);
  11:   
  12:              if (ds == null || ds.Tables.Count == 0)
  13:              {
  14:                  return new EmptyResult();
  15:              }
  16:   
  17:              DataTable table = ds.Tables[0];
  18:   
  19:              JavaScriptSerializer serializer = new JavaScriptSerializer();
  20:   
  21:              List<object> rowData = new List<object>();
  22:   
  23:              foreach (DataRow row in table.Rows)
  24:              {
  25:                  rowData.Add(row.ItemArray);
  26:              }
  27:   
  28:              var data = new { fields = table.Columns.Cast<DataColumn>().Select(c => c.ColumnName).ToArray(), rowData = rowData.ToArray() };
  29:   
  30:              return Content(string.Format("{0}({1});", callback, serializer.Serialize(data)), "application/json");
  31:          }

You can then use this MVC action like this:

   1:  function getList() {
   2:      $("#test").empty();
   3:      $("#test").append("Fetching data...<br/>");
   4:      $.ajax({
   5:          url: "http://myserver/Home/GetListAsJsonp/?server=" + $("#siteUrl").val() + "&listName=" + $("#listName").val() + "&rowLimit=5",
   6:          type: 'GET',
   7:          dataType: "jsonp",
   8:          success: testResult,
   9:          error: function (err, msg) { alert(msg); }
  10:      });
  11:  }
  12:   
  13:  function testResult(data) {
  14:      $("#test").append("Got data.<br/>");
  15:   
  16:      var output = "<table>";
  17:      
  18:      output = output + "<tr>"
  19:   
  20:      for (i in data.fields) {
  21:          output = output + "<th>";
  22:          output = output + data.fields[i];
  23:          output = output + "</th>";
  24:      }
  25:   
  26:      for (index in data.rowData) {
  27:          
  28:          output = output + "<tr>";
  29:          
  30:          for (i in data.fields) {
  31:              output = output + "<td>";
  32:              output = output + data.rowData[index][i];
  33:              output = output + "</td>";
  34:          }
  35:   
  36:          output = output + "</tr>";
  37:      }
  38:   
  39:      output = output + "</table>";
  40:   
  41:      $("#test").append(output);
  42:  }

Now when I say these are the best methods ever written, that doesn't mean this is the best code. It's just very useful in my opinion.  Hope you agree.