Liam McLennan

Thoughts on a Sinatra-like DSL for Server-Side Programming with CoffeeScript

I read somewhere that Kent Beck likes to apply a strategy of taking a technique that works and exaggerating it, to see if the results are likewise exaggerated. He found that specifying the behaviour of components prior to implementation, and in small increments, produced good design, so he exaggerated it and now we have TDD. He found that strong engineering practices, rich communication and delayed decision making was a good way to run projects, so he exaggerated it and now we have Extreme Programming.

In homage to Kent Beck and his work I decided to apply his technique to web frameworks, taking the things that I like and exaggerating them. Here are some of the things that I value in a web framework:

• a rich language
• simplicity
• minimalism
• expressiveness
• opinionatedness

Using the above guidelines I sketched out an idea for a server-side framework for CoffeeScript. It is heavily inspired by Sinatra, with a few minor tweaks and coverted to CoffeeScript. Here is what a controller might look like:

# The first parameter is the route. Like sinatra, there is no route abstraction.
# The second parameter is a method to execute when the route is matched.
# Enforced one-model-in. No need for params[]

get '/', (input_model) ->	
	"Hello World"
	
# Views are found based on the name of the model passed to the view method.
# ? here is the CoffeeScript existential operator.

get 'address/:slug', (address_request) ->
	if address_request?
		@address = address_finder.find_by_slug address_request.slug			
		view @address	
	
# View model objects are automatically given bind_to() and update() 
# methods for binding to and from domain objects.

post 'address/create', (address_model) ->
	@address: address_model.update new Address()
	# do something with @address
	@address_list = address_finder.all()
	view @address_list

As promised the implementation described has some strong opinions: one-model-in, binding methods on view models and no route abstraction.

One-Model-In

One-model-in is a rule that no action may have more than one parameter. All request values (querystring, form, route) are mapped onto the input view model.

Binding Methods on View Models

The input view model objects are automatically given bind_to() and update() methods. By extending the base view model class developers can create more sophisticated view models.

view_model_object.bind_to(domain_object)

Bind to copies the attributes of the input domain object to the view model object.

view_model_object.update(domain_object)

Update copies the attributes of the view model object to the domain model object.

No Route Abstraction

Most MVC web frameworks have an abstraction for routes, meaning that a url is mapped to a route and the route is mapped to a controller action.

Url <—> Route <—> Action <—>

Sinatra simplifies by omitting the route abstraction, and I have too. The advantage is simplicity, the disadvantage is that refactoring an application’s URL scheme might be more difficult.

KeyRef Progress Update

Here are the original mockups for KeyRef

and the implemented screens look like (apologies for the ugliness):

both the application search, and the shortcut search, are working.

I tweaked the application shortcuts page url, so now the url for visual studio shortcuts is:

http://localhost:3000/app/show/4c19e8a5cfbfb01794000001/Visual%20Studio-shortcuts

Also, I have a page for entering new shortcuts that looks like:

Now I can start adding shortcuts, which will be a big job. I’d like to get it online so that people can add shortcuts for their favourite applications.

.NET Configuration Service

My rule of thumb for storing configuration settings is:

• if it does not need to change hard code it
• if it needs to change but the user of the application can’t change it, store it in the configuration file
• if the user can change it, store it in a database

This post discusses configuration data stored in the configuration file.

To make my applications robust and testable I like to write a service that wraps appSettings. What I mean by robust is that it provides useful error data in the event that the application depends upon configuration data that is missing. My configuration service throws the following exception if an appSetting is missing:

A nice clear error message helps me to rapidly diagnose the problem.

This service is something that I have written many times. To be honest, there is often some copy and paste involved. Here is what the most recent incarnation looks like:

    public interface IConfigService
    {
        string ForumURLRoot();
        int EmailSendingRetries();
    }

    public class ConfigService : IConfigService
    {
        // an example configuration element
        public string ForumURLRoot()
        {
            return ReadSetting("ForumURLRoot");
        }
				
        // an example with a cast
        public int EmailSendingRetries()
        {
            return (int)ReadSetting("EmailSendingRetries");
        }

        private string ReadSetting(string key)
        {
            CheckForKey(key);
            return ConfigurationManager.AppSettings[key];
        }

        private void CheckForKey(string appSettingKey)
        {
            if (ConfigurationManager.AppSettings[appSettingKey] == null 
                || ConfigurationManager.AppSettings[appSettingKey].Equals(string.Empty))
            {
                throw new ConfigurationException("Missing appsetting: " + appSettingKey);
            }
        }
    }

Providing a method for each configuration setting gives me:

• the ability to cast if the setting needs to be something other than string
• the ability to easily mock configuration settings for testing

Stubbing a setting, such as ForumURLRoot, is easily accomplished with Moq:

[Test]
public void ConfigMock()
{
   var configMock = new Mock<IConfigService>();
   configMock
       .Expect(config => config.ForumURLRoot())
       .Returns("http://google.com.au");
   Assert.AreEqual("http://google.com.au", configMock.Object.ForumURLRoot());
}

I write this post to expose my ignorance. If I am doing to wrong, let me know.

Lessons Building KeyRef (a .NET developer learning Rails)

Just because I like to build things, and I like to learn, I have been working on a keyboard shortcut reference site. I am using this as an opportunity to improve my ruby and rails skills.

The first few days were frustrating. Perhaps the learning curve of all the fun new toys was a bit excessive. Finally tonight things have really started to come together. I still don’t understand the rails built-in testing support but I will get there.

Interesting Things I Learned Tonight

RubyMine IDE

Tonight I switched to RubyMine instead of my usual Notepad++. I suspect RubyMine is a powerful tool if you know how to use it – but I don’t. At the moment it gives me errors about some gems not being activated. This is another one of those things that I will get to. I have also noticed that the editor functions significantly differently to the editors I am used to. For example, in visual studio and notepad++ if you place the cursor at the start of a line and press left arrow the cursor is sent to the end of the previous line. In RubyMine nothing happens.

Haml

Haml is my favourite view engine. For my .NET work I have been using its non-union Mexican CLR equivalent – nHaml.

Multiple CSS Classes

To define a div with more than one css class haml lets you chain them together with a ‘.’, such as:

.span-6.search_result contents of the div go here

Indent Consistency

I also learnt tonight that both haml and nhaml complain if you are not consistent about indenting. As a consequence of the move from notepad++ to RubyMine my haml views ended up with some tab indenting and some space indenting. For the view to render all of the indents within a view must be consistent.

Sorting Arrays

I guessed that ruby would be able to sort an array alphabetically by a property of the elements so my first attempt was:

Application.all.sort {|app| app.name}

which does not work. You have to supply a comparer (much like .NET). The correct sort is:

Application.all.sort {|a,b| a.name.downcase <=> b.name.downcase}

MongoMapper Find by Id

Since document databases are just fancy key-value stores it is essential to be able to easily search for a document by its id. This functionality is so intrinsic that it seems that the MongoMapper author did not bother to document it. To search by id simply pass the id to the find method:

Application.find(‘4c19e8facfbfb01794000002’)

Rails And CoffeeScript

I am a big fan of CoffeeScript so integrating it into this application is high on my priorities. My first thought was to copy Dr Nic’s strategy. Unfortunately, I did not get past step 1. Install Node.js. I am doing my development on Windows and node is unix only. I looked around for a solution but eventually had to concede defeat… for now.

Quicksearch

The front page of the application I am building displays a list of applications.

When the user types in the search box I want to reduce the list of applications to match their search. A quick googlebing turned up quicksearch, a jquery plugin. You simply tell quicksearch where to get its input (the search textbox) and the list of items to filter (the divs containing the names of applications) and it just works. Here is the code:

$('#app_search').quicksearch('.search_result');

Summary

I have had a productive evening. The app now displays a list of applications, allows them to be sorted and links through to an application page when an application is selected. Next on the list is to display the set of keyboard shortcuts for an application.

Ad-hoc String Manipulation With Visual Studio

Visual studio supports relatively advanced string manipulation via the ‘Quick Replace’ dialog.

Today I had a requirement to modify some html, replacing line breaks with unordered list items. For example, I need to convert:

Infrastructure<br/>
Energy<br/>
Industrial development<br/>
Urban growth<br/>
Water<br/>
Food security<br/>

to:

<li>Infrastructure</li>
<li>Energy</li>
<li>Industrial development</li>
<li>Urban growth</li>
<li>Water</li>
<li>Food security</li>

This cannot be done with a simple search-and-replace but it can be done using the Quick Replace regular expression support. To use regular expressions expand ‘Find Options’, check ‘Use:’ and select ‘Regular Expressions’

Typically, Visual Studio regular expressions use a different syntax to every other regular expression engine. We need to use a capturing group to grab the text of each line so that it can be included in the replacement. The syntax for a capturing group is to replace the part of the expression to be captured with { and }. So my regular expression:

{.*}\<br/\>

means capture all the characters before <br/>. Note that < and > have to be escaped with \.

In the replacement expression we can use \1 to insert the previously captured text. If the search expression had a second capturing group then its text would be available in \2 and so on.

Visual Studio’s quick replace feature can be scoped to a selection, the current document, all open documents or every document in the current solution.

.NET CoffeeScript Handler

After more time than I care to admit I have finally released a rudimentary Http Handler for serving compiled CoffeeScript from Asp.Net applications.

It was a long and painful road but I am glad to finally have a usable strategy for client-side scripting in CoffeeScript.

Why CoffeeScript?

As Douglas Crockford discussed in detail, Javascript is a mixture of good and bad features.

The genius of CoffeeScript is to treat javascript in the browser as a virtual machine.

By compiling to javascript CoffeeScript gets a clean slate to re-implement syntax, taking the best of javascript and ruby and combining them into a beautiful scripting language. The only limitation is that CoffeeScript cannot do anything that javascript cannot do.

Here is an example from the CoffeeScript website. First, the coffeescript syntax:

reverse: (string) ->
  string.split('').reverse().join ''

alert reverse '.eeffoC yrT'

and the javascript that it compiles to:

var reverse;
reverse = function(string) {
  return string.split('').reverse().join('');
};
alert(reverse('.eeffoC yrT'));

Areas For Improvement ;)

The current implementation is deeply flawed, however, at this point I’m just glad it works. When the server receives a request for a coffeescript file the following things happen:

1. The CoffeeScriptHandler is invoked
2. If the script has previously been compiled then the compiled version is returned.
3. Else it writes a script file containing the CoffeeScript compiler and the requested coffee script
4. The process shells out to CScript.exe to to execute the script.
5. The resulting javascript is sent back to the browser.

This outlandish process is necessary because I could not find a way to directly execute the coffeescript compiler from .NET. If anyone can help out with that I would appreciate it.

Photos from MS Webcamps Sydney

View Full Album

KeyRef – A Keyboard Shortcut Reference Site

The mouse is like computer training wheels. It makes using a computer easier – but it slows you down.

Like many of my peers I am making a effort to learn keyboard shortcuts to reduce my dependence on the mouse. So I have started accumulating browser bookmarks to websites listing keyboard shortcuts for vim and resharper etc.

Based on the assumption that I am not the only person who finds this untenable I am considering building the ultimate keyboard shortcut reference site. This is an opportunity for me to improve my rails skills and hopefully contribute something useful to the anti-mouse community.

Mockups

Shortcuts will be grouped by application, so the first thing a user needs to do is find their application. They do this by typing the application name into a textbox and then selecting from a reducing list of applications. This interface will work like the stackoverflow tags page.

Selecting an application will take the user to a page that lists the shortcuts for that application. This page will have a permalink for bookmarking. Shortcuts can be searched by keyword or by using the shortcut.

Managing .NET Deployment Configuration With Rake

Rake is a ruby internal DSL for build scripting. With (or without) the help of albacore rake makes an excellent build scripting tool for .NET projects.

The albacore documentation does a good job of explaining how to build solutions with rake but there is nothing to assist with another common build task – updating configuration files.

The following ruby script provides some helper methods for performing common configuration changes that are required as part of a build process. 

class ConfigTasks

	def self.set_app_setting(config_file, key, value)
		ovsd_element = config_file.root.elements['appSettings'].get_elements("add[@key='#{key}']")[0]
		ovsd_element.attributes['value'] = value
	end
	
	def self.set_connection_string(config_file, name, connection_string)
		conn_string_element = config_file.root.elements['connectionStrings'].get_elements("add[@name='#{name}']")[0]
		conn_string_element.attributes['connectionString'] = connection_string
	end
	
	def self.set_debug_compilation(config_file, debug_compilation)
		compilation_element = config_file.root.elements['system.web'].get_elements("compilation")[0]
		compilation_element.attributes['debug'] = false
	end
	
	private
	
	def self.write_xml_to_file(xml_document, file)
		File.open(file, 'w') do |config_file|
			formatter = REXML::Formatters::Default.new
			formatter.write(xml_document, config_file)
		end
	end

end

To use, require the file and call the class methods, passing the configuration file name and any other parameters.

require 'config_tasks'

ConfigTasks.set_app_setting 'web.config', 'enableCache', 'false'

Funny Captcha

I like this capture because it reminds me of the ‘England Prevails’ dialog from V for Vendetta.

Less is more: The making of a 37 signals style pizza

For years now we have been hearing from 37 signals that the way to bake a great web app is to build less – well the same is true of pizza.

Our western hedonism has led us to pursue ever cheesier and more stuffed crusts at the expense of the simple flavours. All we are left with is a fatty, salty heart attack in waiting.

The Italians know that the secret to great taste is simplicity. With that in mind I decided to base my pizza masterpiece on these simple flavours:

• tomato
• sopressa (spicy aged salami)
• mozzarella
• garlic
• basil

Of course the first thing one needs when making pizza is a base.

A freshly made base is extremely important but unfortunately I was too lazy.

Next up is the tomato sauce. My wife made the sauce by reducing some tomatoes and adding herbs and sugar.

We had selected some fine ingredients to make our topping: sopressa salami, fresh basil and the best mozzarella we could find.

It is, according to google, important to bake pizza at a high temperature, so I set the oven to 250C (480F).

Here are the before and after shots:

Meanwhile, the dog did nothing.

Console 2: An upgraded windows console

Lately I have been using the windows console a lot. I find that I often need a number of console windows open at once. The regular windows console does not handle this well.

Console2 is a more advanced console for windows. It has a tabbed interface and a number of other nice features. It supports alpha transparency if you have Mac envy, it has improved text selection and copy/paste and it is far more customizable than the default console.

If you look in the background of the above image you can see this post. Now you know what the matrix is.

Twitter Customer Sentiment Analysis

The breakable toy that I am currently working on is a twitter customer sentiment analyser.

It scrapes twitter for tweets relating to a particular organisation, applies a machine learning algorithm to determine if the content of tweet is positive or negative, and generates reports of the sentiment data over time, correlated to dates, events and news feeds.

I’m having lots of fun building this, but I would also like to learn if there is a market for quantified sentiment data. So that I can start to show people what I have in mind I have created a mockup of the simplest and most important report. It shows customer sentiment over time, with important events highlighted. As the user moves their mouse to the right (forward in time) the source data area scrolls up to display the tweets from that time. The tweets are colour coded based on sentiment rating.

After I started working on this project I discovered that a team of students have already built something similar. It is a lot of fun to enter your employers name and see what it says.

Random MongoDb Syntax: Updates

I have a MongoDb collection called tweets. Each document has a property system_classification. If the value of system_classification is ‘+’ I want to change it to ‘positive’.

For a regular relational database the query would be:

update tweets
set system_classification = 'positive'
where system_classification = '+'

the MongoDb equivalent is:

 db.tweets.update({system_classification: '+'}, 
 {$set: {system_classification:'positive'}}, false, true)

Refactoring an ERB Template to Haml

ERB is the default view templating system used by Ruby on Rails. Haml is an alternative templating system that uses whitespace to represent document structure. The example from the haml website shows the following equivalent markup:

#profile
  .left.column
    #date= print_date
    #address= current_user.address
  .right.column
    #email= current_user.email
    #bio= current_user.bio

<div id="profile">
  <div class="left column">
    <div id="date"><%= print_date %></div>
    <div id="address"><%= current_user.address %></div>
  </div>
  <div class="right column">
    <div id="email"><%= current_user.email %></div>
    <div id="bio"><%= current_user.bio %></div>
  </div>
</div>

I like haml because it is concise and the significant whitespace makes it easy to see the structure at a glance.

This post is about a ruby project but nhaml makes haml available for asp.net MVC also.

The ERB Template

Today I spent some time refactoring an ERB template to Haml. The template is called list.html.erb and its purpose is to render a list of tweets (twitter messages).

<style>
form { float: left; }
</style>

<h1>Tweets</h1>

	<table>
		<thead><tr><th></th><th>System</th><th>Human</th><th></th></tr></thead>

<% @tweets.each do |tweet| %>

	<tr>
		<td><%= h(tweet['text']) %></td>
		<td><%= h(tweet['system_classification']) %></td>
		<td><%= h(tweet['human_classification']) %></td>
		<td><form action="/tweet/rate" method="post">
			<%= token_tag %>
			<input type="submit" value="Positive"/>
			<input type="hidden" value="<%= tweet['id']%>" name="id" />
			<input type="hidden" value="positive" name="rating" />
		</form>
		<form action="/tweet/rate" method="post">
			<%= token_tag %>
			<input type="submit" value="Neutral"/>
			<input type="hidden" value="<%= tweet['id']%>" name="id" />
			<input type="hidden" value="neutral" name="rating" />
		</form>
		<form action="/tweet/rate" method="post">
			<%= token_tag %>
			<input type="submit" value="Negative"/>
			<input type="hidden" value="<%= tweet['id']%>" name="id" />
			<input type="hidden" value="negative" name="rating" />
		</form>
		</td>
	</tr>

<% end %>

	</table>

Haml Template: Take 1

My first step was to convert this page to a Haml template in place. Directly translating the ERB template to Haml resulted in:

list.haml

%style form {float: left;}

%h1 Tweets

%table
	%thead
		%tr
			%th
			%th System
			%th Human
			%th
	%tbody
	- @tweets.each do |tweet|		
		%tr
			%td= tweet['text']
			%td= tweet['system_classification']
			%td= tweet['human_classification']
			%td
				%form{ :action=>"/tweet/rate", :method=>"post"}
					= token_tag
					<input type="submit" value="Positive"/>
					<input type="hidden" value="positive" name="rating" />
					%input{ :type=>"hidden", :value => tweet['id']}
				%form{ :action=>"/tweet/rate", :method=>"post"}
					= token_tag
					<input type="submit" value="Neutral"/>
					<input type="hidden" value="neutral" name="rating" />
					%input{ :type=>"hidden", :value => tweet['id']}
				%form{ :action=>"/tweet/rate", :method=>"post"}
					= token_tag
					<input type="submit" value="Negative"/>
					<input type="hidden" value="negative" name="rating" />
					%input{ :type=>"hidden", :value => tweet['id']}
	end

I like this better already but I can go further.

Haml Template: Take 2

The haml documentation says to avoid using iterators so I introduced a partial template (_tweet.haml) as the template to render a single tweet.

_tweet.haml

%tr
	%td= tweet['text']
	%td= tweet['system_classification']
	%td= tweet['human_classification']
	%td
		%form{ :action=>"/tweet/rate", :method=>"post"}
			= token_tag
			<input type="submit" value="Positive"/>
			<input type="hidden" value="positive" name="rating" />
			%input{ :type=>"hidden", :value => tweet['id']}
		%form{ :action=>"/tweet/rate", :method=>"post"}
			= token_tag
			<input type="submit" value="Neutral"/>
			<input type="hidden" value="neutral" name="rating" />
			%input{ :type=>"hidden", :value => tweet['id']}
		%form{ :action=>"/tweet/rate", :method=>"post"}
			= token_tag
			<input type="submit" value="Negative"/>
			<input type="hidden" value="negative" name="rating" />
			%input{ :type=>"hidden", :value => tweet['id']}

and the list template is simplified to:

list.haml

%style form {float: left;}

%h1 Tweets

%table
    %thead
        %tr
            %th
            %th System
            %th Human
            %th
    %tbody
        = render(:partial => "tweet", :collection => @tweets)

That is definitely an improvement, but then I noticed that _tweet.haml contains three form tags that are nearly identical.

Haml Template: Take 3

My first attempt, later aborted, was to use a helper to remove the duplication. A much better solution is to use another partial. 

_rate_button.haml

%form{ :action=>"/tweet/rate", :method=>"post"}
	= token_tag
	%input{ :type => "submit", :value => rate_button[:rating].capitalize }	
	%input{ :type => "hidden", :value => rate_button[:rating], :name => 'rating' }
	%input{ :type => "hidden", :value => rate_button[:id], :name => 'id' }

and the tweet template is now simpler:

_tweet.haml

%tr
	%td= tweet['text']
	%td= tweet['system_classification']
	%td= tweet['human_classification']
	%td
		= render( :partial => 'rate_button', :object => {:rating=>'positive', :id=> tweet['id']})
		= render( :partial => 'rate_button', :object => {:rating=>'neutral', :id=> tweet['id']})
		= render( :partial => 'rate_button', :object => {:rating=>'negative', :id=> tweet['id']})

list.haml remains unchanged.

Summary

I am extremely happy with the switch. No doubt there are further improvements that I can make, but I feel like what I have now is clean and well factored.