Back in June, I posted my own version of the Joel Test – updated for 2010. [See Here.]

1. Do you have a change management system?

2. Can everyone make a build in one step?

3. Do your daily builds include automated tests?

4. Is work item tracking integrated with source control?

5. Do you fix bugs and write new code?

6. Do you track progress and manage change?

7. Do you have a requirements management system?

8. Do programmers have quiet working conditions and teaming rooms?

9. Do you use the best tools money can buy?

10. Are your testers involved in requirements management?

11. Do new candidates review code during their interview?

12. Do you do hallway usability testing?

I know I’m late to the iPhone development party, and consequently I’m also late to the Objective-C debates. But I can’t help but chime in; so here goes . . .

Back in the Day

Back in the C programming days (while I was apparently sleeping), there seem to have been two directions: The C++ road, and the objective-C road. I have never seen Objective-C until OS-X, and I have never worked with it until a couple of weeks ago, but there is certainly a lot of goodness going on here – much more than C++ ever attempted. In hindsight, I can say that every large C++ project I’ve seen has grown into a mess – that includes apps using UI frameworks, as well as apps using STL, boost, and other frameworks and libraries. Objective-C does show its age from time to time, but it contains many things I mistakenly thought were “invented” later (probably invented earlier – even before smalltalk – I’ll have to do some research sometime). Here are a few thoughts from an absolute beginner in Objective C. It should be fun to read this a few months from now and reflect upon my initial impressions.

Learn a New Language Every Year

This advice from the Pragmatic Programmer is good advice, even if it is not really possible to follow every year. The advice is given so that you may learn a new way to think.

What I am learning to think about:

• Static typing and dynamic dispatch (and dynamic typing, too, if you want)
• Message passing vs method calling, including ignoring and forwarding messages
• Protocols vs Java/C# interfaces
• Categories to add (and replace) methods on a class
• Newer features, such as Properties, garbage collection

Some of the silliness

There still remains some silliness, like header files, much of the syntax, and the lack of namespaces (I am getting tired of prefixing all of the NextStep, err, I mean Cocoa APIs with ‘NS’), but that’s a result of the time period when all this stuff was developed, and the close relationship to the ‘C’ language, which is also touted as a feature but I could care less about. But all of this is very minor.

Overall, a much better extension of C than C++ in almost every way.

2009 Edition.

No. 1 Solid State Disk

The crucial CT64M225 64GB ($234) or CT128M225 128GB ($429), or the ultimate Intel X25 160GB ($549) for the notebook machine. Improve performance, improve battery life. Best of all, you don’t have to wait a few seconds after closing the lid for the HDD to spin down before you pick up and go.

No. 2 Roku Player

The Roku now has tech video podcasts available, as well as netflix and Amazon video on demand. All it needs is “Play To” DLNA capability so we can stream everything from our Win7 computers to it.

No. 3 Stand-Alone Blue Ray Player

Forget about the playstation 3, that just sucks up your time playing games. Purchase the LG BD390 instead (or equivalent player with DLNA support) – not for playing blu-ray disks, as blu-ray will fade away before it ever gets started (physical disk delivery will disappear), but rather as a DLNA receiver for both audio and video, a YouTube box, and a netflix online device. Now all we need is support for Hulu, . . . [PlayTo, not play]

No. 4 Netflix Subscription

We need some content for our #2 and #3 video/audio devices (besides everything already on our computers).

No. 5 Buy another Monitor

You can always use another monitor, and screen real estate is king. Make this one 1920x1080 pixels widescreen, good quality, and plunk it down in the middle of your current setup.

No. 6 Stocking Stuffers

Grab a bunch of 4GB USB sticks for Windows 7 and Server 2008r2 boot media, as well as some memory and disk checking utilities. You’ll need at least 4 of these, at about $11 a piece. Optical disks are not worth the shelf space. Throw in an all-in-one USB stick adapter for your SD cards and the phone’s microSD, and you’re all set wherever you go.

No. 7 MacOS X Machine

Re-discover your unix roots, learn a new language (objective-C), and look at all the pretty, shiny things. Try for a MacBook Pro refurbished (or ebay) to make to reasonably affordable, or get a Mac Mini and hook up one of your monitors from time-to-time. Don’t forget to install Windows 7 – get a 1TB drive and split it in half. No virtual machines needed, here.

No. 8 MiFi portable Wi-Fi

Assuming you have a data plan, keep the whole family happy in the car on the next trip. Don’t expect to watch a movie on Hulu, however, because it’s just not going to happen. But everyone can check their email - and tweet. [Are we there, yet?]

No. 9 Kindle 2

They added native PDF and longer battery life (by turning off the wireless when not in use – Duh!) to this little guy, which now makes the list for 2009.

No. 10 Core i5, 16GB RAM, P55 Motherboard

Update and upgrade your main dev box with lots more RAM for those virtual machines using the best-value Core i5 quad chip. You won’t heat up the room, just the code.

Merry Christmas.

New Apple iMac Arrives

The latest edition to the growing home computing center is an Apple iMac (Intel E7600 Core 2 Duo/8Gb RAM/1Tb HD/21.5"), which I promptly setup to use Boot Camp with Windows 7 as the default OS. This makes a great Family Room machine, in Windows or MacOS (despite missing HDMI port), and doubles as the iPhone development platform. What is more interesting is the purchase survey, which I took today.

Apple New Purchase Survey

Going though the questions it became clear that there were 3 main goals to the survey: canvass the effect of recent advertising campaigns; perform some basic market research; and make some decisions about Macs of the future. I answered the question about “who am I” correctly (software engineer), so I am sure they ignored everything I said with respect to advertising. Questions related to the second goal were interesting to me (do most people have wireless N or wireless G?), but the third goal – that’s the important one. We, as an industry and as consumers, have benefited from Apple’s presence in the market and I would like this role to continue. I want a say about Apple’s products of the future!

So, just in case my entire survey response ended up in the bit-bucket, I will reiterate some of my survey responses here in the vein hopes that they will actually be heard. . .

What Apple Did Right

• Use Intel hardware

The use of industry standard hardware – not just Intel processors – saved Apple from death. This includes seamless operation with most peripherals (printers, etc.) and protocols in use today and not just processors, chipsets, and video cards. But most important is x86/x64, and the ability to run Windows on Mac hardware.

• Build what people want

A few misses over the years, but mostly Apple builds what people want.

• Provide Boot Camp (boot manager and drivers)

If your going to spend extra on a Mac, you might as well get something extra in return. Such as the ability to run Windows and MacOS on the same box. Nice.

• Great hardware design. I mean, great hardware design.

What Apple Still Does not “Get”

• The menu bar has got to go

So what if the original MacOS had a bar at the top. MultiFinder is gone;  so get rid of the bar.

• Secondary mouse click is critical (not to mention useful)

The new iMac comes with an excellent multi-touch wireless mouse that can detect if you click on its left side vs its right side. By default, however, the mouse control panel preferences are setup to ignore this feature. Although it was a simple change to enable what Apple calls secondary click, this should be the default.

• Charge a little less. Please?

I wish I could have gotten an i5 (or i7) for the price, and adding $200 for 4Gb of DDR3/1066 RAM is double the price I paid from Crucial (which is where I got my second half). I won’t mention what an additional TB of disk space costs, but at least memory is user-replaceable.

• No removable drive? Really?

Somebody tell Apple that (a) hard drives fail, (b) capacity requirements always go up over time, (c) costs always come down over time, and (d) new technology is just around the corner (SSD). Creating a computer without a user-replaceable drive is just stupid. In a year or so when I pry apart the glue in the beautiful glass display panel with a flat-blade screwdriver in order to access the drive, I’ll be cursing out loud. Drop the velvet lint cloth and add 2 screws and a tray.

• Development Tools

There’s good and bad, here. XCode is free, and comes with the OS. The App Store is obviously popular (and effective) with consumers. But objective-C? Really? Didn’t even take the time to remove the “ns” (NextStep) from the framework names! Focusing on what’s important or too lazy . . .not sure.

But then there are all those users. Waiting for your app.

My first thought – who needs another language? Are new languages the new rage?

If I were to design a new language (I am not qualified, trust me), what would it address . . .

• It would be garbage collected (no brainer)
• It would run in a VM (actually, many different VMs )
• It would also be statically typed (I think many would disagree these days)
• Threading would be a focus (with message passing as a language feature). Certainly include some aspects of functional programming languages
• It would have first class functions to throw around
• Some sort of cool, built-in data types – if only to set it apart from other languages.
• Nearly all functionality in libraries, and try to get others to actually write the libraries

Sounds a bit like Google Go - “Expressive, concurrent, garbage collected” they say. Fast, safe, concurrent, fun, open source they continue. Fun and fast? I don’t think “compile time” is an issue and nearly everything these days, interpreted or not, runs “nearly as quickly as C or C++ code”. So that’s nothing new. All programming is fun. (Well, nearly all).

My language would not have a googly-goofy name, like “Go” with all the “Go nuts”, “Go for it”, etc., phrases being thrown around. I do like the co-routines moniker, err, I mean Go-routines, however. At least they did not call it “goo” or “goog”. In fact, without the goroutines I probably would not have given this thing a second look.

So what is new with this language?

A Systems Programming Language

I think that title is a clue as to what this all might be about – I would love to see C/C++ disappear, but I would like to see that done through evolution of those languages. Something like removing all the “bad goo” from C/C++, but leaving the good stuff alone. Maybe that is impossible – we can only pile on, not take away. A colleague of mine once said that C/C++ would always be around because someone has to write the thread scheduler and the device drivers for the OS – I disagree, eventually we will jettison what is bad about C and keep what’s not. Is that GO? I don’t know.

Why does it have pointers? Who in their right mind would add that as a language feature? It has no pointer arithmetic – so maybe this is OK, then? Slices – they seem to allow you to address arbitrary blobs of bytes (defined via a pointer, maybe) or portions of arrays, as the documentation states, in a semi-safe manner without the pointer arithmetic. Sounds reasonable for a systems programming language. I wonder, then, why are strings immutable?

Things I like

• Maps, slices, and channels
• Go-Routines
• No implicit numeric conversions
• Lots of implicit “compatible conversions” (my terminology)
• No side-effects or alternatives syntaxes for things
• No operator overloading

Things Not-so-hot

• No exception handling
• No generics
• Pointers
• No method overloading

No type inheritance? I’ll have to think about that one for a while. Sounds very interesting, though, since a significant amount of refactoring I seem to do involves unwinding complex type hierarchies into composition models. There is also implicitly-inherited (my terminology) types – i.e., when one type just happens to implement a set of methods that matches another type or interface. They are then magically, implicitly compatible. That is cool. Very cool.

I plan to spend some time over the next few months looking at Google Go. There are lots of very smart people over in google-land, so I am sure there is a lot to consider and learn from the ideas around Go. Too bad I’ll have to fire up linux or actually boot the Macbook into OSX instead of Windows 7 to play with it. Stupid decision on google’s part, if you ask me.

One of the things I have always hated was “Windows Search”. Why the heck would I need “instant results” – instant as in donating countless CPU cycles and disk churns to background indexing operations running all day long on my box? YUK! I am never actually “looking” for something – I know where everything is. It’s right where I put it. Who are these people “looking” for things on their computers all the time?

Typical Scenario

My typical use case came up again this morning: I downloaded a source distribution from Google Code, and decided to check the files into my local SCM system (sidebar: this project is not taking any patches). Before doing this, I would like to delete all of the hidden “.svn” folders that came down with the distro. “Search for all folders named .svn”. Now keep in mind that I am not actually looking for the folders, as I know where they are (there is one folder named .svn in each of the folders in the distribution). I simply want to find them so I can control-A (select all) and press the delete key. You see what I mean.

Trouble with Windows Search

I always have trouble with Windows Search. I start typing “.svn” and immediately my file display window changes and I get a bunch of wrong answers – files with the letters “svn” in them (dot being ignored), sometimes files with “svn” inside of them, as well as virtually all of the files inside the folders, since many contain .svn extensions. Great. Hundreds of results when I was expecting around ten. What happened to the good old days when “find files” actually did what I wanted!

Read the Manual

Read the manual. For the first time, I decided to read the manual. I found a page at microsoft.com with the search syntax. For my scenario:

kind:folder .svn

The dot “.” is still ignored, and you get all folders – not just folders with “svn” in the name. Hmm. A little more digging. . .

kind:folder name:.svn

Now I am getting just the folders named “.svn”, but I am still getting folders named “svn” (no dot), as well as any folder with “svn” in the name. Hmm.

kind:folder name: “.svn”

That did it. The dot is still being ignored, despite the quotes, but it is not a wildcard in some sort of pseudo-regular expression. The documentation says that * and ? are the wildcard characters, taking on their typical meanings. There’s plenty of syntax for AND, OR, greater or less than (for things like dates and sizes), and virtually any property you can think of – in or out of the file(s).

Maybe I’ll spend some more time looking over the documentation. Maybe not. At least I can now find a bunch of folders to delete. The next time I’m looking for a MSWord document containing the phrase “place contract”, the word “lease”, and was edited within the last 3 days, created by “mydomain\joe”, and last saved by “otherdomain\mike” – I’ll know how to do it. Thanks, Microsoft.

Visual Studio 2010 beta2 + Framework 4.0

. . . with a few comments on Windows Virtual PC.

Installation

I don’t “work” on my laptop – mostly used for travel, library, watching Hulu and channel 9, etc. Since I will be out of town for a while, however, I decided to install VS20101b2 in a winVPC for the trip on the laptop. The only complaint about the new VPC offering from Microsoft is that when it is running, it takes forever for my notebook to initiate sleep mode. I don’t recall the old product having this issue.

Installation of 2010 was a breeze, especially Team Foundation Server (holy crap is that a different experience!). The new icon needs work – looks very bad in the new Windows 7 taskbar with the default theme.

First Impressions

The first thing I did was fire up the VM and connect to TFS server – all running on the laptop – in a 1GB VM, I might add. I create a new team project (‘TestProject’). This all worked. I seem to remember several days before I was able to accomplish this on tfs2005, and at least a few hours on tfs2008. Today’s elapsed time is a couple of minutes (for the TFS part).

Overall, I found a snappy IDE that has a finished feel to it. Nice. March release should be no problem.

Branching and Merging

I setup the TestProject with a typical branching and merging strategy (dev/main/prod), added a few work items and a bug, and got to work.

[ Note: I like the little ‘Save’ icon – saves a picture of the visualization] There seem to be lots of tools accessible from this UI, such as branch compare tools. Overall, this visualizer is a nice feature (the simple picture shown does not do justice). I’m still looking for the merge visualizations (update: found them!). I’d like to see merge arrows that are labeled per the changeset description and clickable to show history/changeset details for merges. It looks like I’ve got some exploring to do.

Team Web Access

I was not expecting this. Even in the Basic product running on a local client OS. Nice touch. It’s a beautiful source, work item, project browser web application. Who needs SharePoint?

New in Framework

I’ve already blogged a few times about changes in the Framework 4.0 beta. A few more in this release:

• DLR library
• Several more threading primitives
• System.Collections.Concurrent namespace
• MEF improvements
• In-process side-x-side CLR

Static Reflection in .Net

Where have I been?

I recently took a few minutes to look through the latest MSDN magazine and ran across a nice article by Jeremy Miller on “Functional Programming for Everyday .NET Development”. The section titled “Lambdas as Data” was my first introduction to static reflection in .Net. Where the heck have I been? I don’t write database persistence code, so I am not familiar with NHibernate (or fluent NHibernate) (which is where Jeremy’s sample comes from), but I am still surprised at how little I understand about everything Linq brought to the .Net party. A quick search around the internet reveals this topic is well-covered, and several years old.

Example Code

Suppose you have a need to dynamically discover properties on an object. Easy, right? After all, that’s what the System.Reflection namespaces is for! However..

Let’s look at a class named “SomeObject” with a property “SomeProperty”:

class SomeObject

{

    public string SomeProperty { get; set; }

}

You can use so-called “static reflection” like this:

   1: class Program

   2: {

   3:     static void Main(string[] args)

   4:     {

   5:         SomeObject o = new SomeObject()

   6:         {

   7:             SomeProperty = "Test"

   8:         };

   9:         staticReflection(o, e => e.SomeProperty);

  10:         Console.ReadKey();

  11:     }

  12:     static void staticReflection<T>(T o, Expression<Func<T,object>> expression)

  13:     {

  14:         MemberExpression me = expression.Body as MemberExpression;

  15:         if (me != null && me.Member.MemberType == System.Reflection.MemberTypes.Property)

  16:         {

  17:             Console.WriteLine("Property: '{0}' of type '{1}' has value '{2}'.",

  18:                 me.Member.Name,

  19:                 expression.Body.Type,

  20:                 expression.Compile().Invoke(o)

  21:                 );

  22:         }

  23:     }

  24: }

The output is:

Property: 'SomeProperty' of type 'System.String' has value 'Test'.

Very cool. Now I need to understand why the second argument has to be something like Expression<Func<T, object>> rather than simply Expression<T>. If the latter is used, the compiler rewards me with:

Error    1    Cannot convert lambda to an expression tree whose type argument 'StaticReflection.SomeObject' is not a delegate type

It’s the whole “return type” thing on the delegate that throws me as well – where does that come from? In my code sample I have declared the Func return type as type “object” (it is actually a string in my case). Why don’t I have to do something like “e => {return e.SomeProperty;}”, and how come a lambda expression with a statement body like that cannot be converted into an expression tree (compiler’s words, not mine)?

I still have a little more learning to do before really understanding what’s going on here. My suspicion is that Linq brought a heck of a lot more to the party than it might appear at first look.

Designing Software for Scientists

Creating a software design for scientists can be tricky business. Unlike the businessman, who is better off looking at the pretty database report or marveling over the catchy name of the latest Forms over Data application, the scientist can actually contribute to the implementation – provided you use the right architecture. In year’s past (I’m referring to 20+ years ago), the typical Physicist / Chemist / etc. was nearly on par with the typical software engineer / computer scientists with respect to software design skills. Although the scientific community of that time preferred the ‘GOTO’, they nevertheless understood the value of using subroutines to nearly the same degree as a computer scientists.

Not so, today. Software design has come a long way since then.

Software Design has Matured

Software design has matured. We have terms like inheritance and encapsulation. We talk about singletons, abstract factories, and chain of responsibility. We use terms like polymorphism, covariance, and contravariance (as applied to software design). We compose systems with Inversion of Control containers. Mock objects are used for unit testing. Reflection, dynamic and static languages. The discipline of software engineering has matured to a point where the typical scientist cannot possibly hope to understand what makes a well-design system – no more than a software engineer can understand how to model protein folding of polypeptides.

So how do we design software so that other scientists and engineers can participate in the implementation? Here are some ideas.

1. Keep it Simple.

Don’t compose in an IoC container – too abstract. Do use layers, repositories, and multiple program units. Scientists understand process control, state transitions, and other engineering/scientific/mathematics concepts so use them in your software designs. [Object models? Not sure where I stand on this one. There might be a whole blog post in here somewhere.]

2. Work in Pairs

Match a subject matter expert with a software design expert.

3. Limit Participation

Limit the “coding” performed by the scientist to the functional code, not the design. The infrastructure, layout, program flow as well as all meta-design elements (project structure, build environment, coding standards, etc.) – essentially the software design, should be strictly the realm of the software architect. Leave the algorithms for the subject matter expert.

4. Move the problem domain.

Let the scientists program in their own “language” - provide a scripting component, meta-programming, or data-driven model such that the subject matter expert remains closer to their problem domain, even if this means additional software (such as interpreters, data transformation engines) must be developed. In a really large system, you might go so far as to develop a rules engine or other knowledge base approach.

5. Use Abstractions

The nature of software is to provide abstractions. This also happens to be common of all science and engineering. You can use your entire software design Bag ‘o Tricks provided you supply an appropriate level of abstraction over top. Explain the abstraction, and the typical scientist will have no trouble using it, even if they really don’t know what is going on. So in spite of the first idea listed here, you can fire up your favorite IoC container and use it – as long as you provide the right abstraction for your scientific colleagues.

Three’s a Charm

It looks like the third iteration of Team Foundation Server might be a good one. I just finished reading Brian Harry’s post on TFS 2010 and it contains some interesting references.

SourceSafe Replacement?

I never could understand why TFS was often compared to SourceSafe. All the sessions on migrating from VSS to TFS, TFS for VSS users, etc. Is a team really going to migrate from VSS to TFS? Really? You have got to be kidding me! TFS is change management (to borrow a phrase from Rational), and VSS was barely version control.

… but now I think I get it …

All those VSS (or CVS, etc.) users were left stranded – there was no lightweight version control server for individuals (or very small teams). So they all went to SVN. Quick setup, version control, painful (but possible) branching and merging, command line, nice shell client, poor VS client, and all of that.

VSTS 2010

In steps Visual Studio Team System, Team Foundation Server, 2010.

• Can run locally on Windows 7 and Windows Vista client OS’
• Simple installation / Configuration
• Can optional use SQL Server express edition
• Can be installed on a domain controller
• Stand-alone version includes Work Item Tracking, Version Control

Note that this stand alone version also appears to include the build server/services!

I assume with the new project collections feature, you could move your SQL Express database over to a real SQL Server when your team gets a bit bigger. And good pricing is also on the Redmond minds.

Looks to me like they did their research. And did it well.

Nice.

The Process class in .Net makes it easy to run command line tools from within a .Net program. Here is an implementation with a couple of twists:

• Environment variable expansion
• Optional asynchronous execution
• Transformation of output into an enumerable collection of objects

The implementation takes the form of an object named CommandLineTool with the following constructors:

public CommandLineTool(string command)
    : this(command, String.Empty, Environment.CurrentDirectory, _defaultTransform) { }
public CommandLineTool(string command, string arguments)
    : this(command, arguments, Environment.CurrentDirectory, _defaultTransform) { }
public CommandLineTool(string command, string arguments, string workingDirectory)
    : this(command, arguments, workingDirectory, _defaultTransform) { }
public CommandLineTool(string command, string arguments, string workingDirectory, Func<string, object> transform)
{
    this.Command = Environment.ExpandEnvironmentVariables(command);
    this.Arguments = Environment.ExpandEnvironmentVariables(arguments);
    this.WorkingDirectory = Environment.ExpandEnvironmentVariables(workingDirectory);
    this.Transform = transform;
    _isBusy = false;
}

The simplest form provides for an executable filename only, while the last constructor accepts arguments, working directory, and transformation function for the output lines. Note that environment variables, such as "%ComSpec%, may be included in any of the first three arguments.

Output Transformation

Each line of output (strings) can be sent to a transformation function, defined as a Func<string, object>, and returned as an enumerable collection of objects (IEnumerable<Object>) accessed via the OutputObjects property. Output can also be expressed as a single string (Output) or an array of lines of text (OutputLines). These are exposed as properties. The transformation function is free to return null to exclude an item from the OutputObjects collection.

   1: /// <summary>

   2: /// Command output.

   3: /// </summary>

   4: public string Output { get; private set; }

   5: /// <summary>

   6: /// Command output lines.

   7: /// </summary>

   8: public string [] OutputLines

   9: {

  10:     get

  11:     {

  12:         if (Output != null)

  13:         {

  14:             return Output.Split(_seperators, StringSplitOptions.RemoveEmptyEntries);

  15:         }

  16:         return null;

  17:     }

  18: }

  19: /// <summary>

  20: /// Output Objects.

  21: /// </summary>

  22: /// <remarks>

  23: /// Output lines are converted to output objects using the transform

  24: /// function. The default transform functions results in an enumeration

  25: /// of the out lines (strings).

  26: /// </remarks>

  27: public IEnumerable<Object> OutputObjects

  28: {

  29:     get

  30:     {

  31:         if (Transform != null)

  32:         {

  33:             return OutputLines.Select(Transform).Where(o => o != null);

  34:         }

  35:         return null;

  36:     }

  37: }    

Asynchronous and Synchronous Operation

Both asynchronous and synchronous modes of operation are provided. An event (ExecuteCompleted) is raised following the completion of an asynchronous execution. Overloads are provided for both the Execute() and ExecuteAsync() methods to allow for an alternate transformation function to be applied in successive calls. Only one asynchronous operation at a time is allowed. The asynchronous version simply calls on the synchronous version using a background thread from the thread pool. Error output is provided in a similar manner to standard output, except that there is no option for transformation of lines of text on the error output to objects.

   1: /// <summary>

   2: /// Execute the command.

   3: /// </summary>

   4: /// <param name="transform">Transform to apply to the output.</param>

   5: /// <returns>Command result.</returns>

   6: public int Execute(Func<string, object> transform)

   7: {

   8:     this.Transform = transform;

   9:     return Execute();

  10: }

  11: /// <summary>

  12: /// Executes the command.

  13: /// </summary>

  14: /// <returns>Command result.</returns>

  15: public int Execute()

  16: {

  17:     ProcessStartInfo info = new ProcessStartInfo();

  18:     info.FileName = this.Command;

  19:     info.Arguments = this.Arguments;

  20:     info.WorkingDirectory = this.WorkingDirectory;

  21:     info.CreateNoWindow = true;

  22:     info.UseShellExecute = false;

  23:     info.RedirectStandardError = true;

  24:     info.RedirectStandardOutput = true;

  25:     Process p = Process.Start(info);

  26:     Output = p.StandardOutput.ReadToEnd();

  27:     ErrorOutput = p.StandardError.ReadToEnd();

  28:     return p.ExitCode;

  29: }

  30: /// <summary>

  31: /// Execute the command asynchronously.

  32: /// </summary>

  33: /// <param name="transform">Transform to apply to the output.</param>

  34: public void ExecuteAsync(Func<string, object> transform)

  35: {

  36:     this.Transform = transform;

  37:     ExecuteAsync();

  38: }

  39: /// <summary>

  40: /// Execute the command asynchronously.

  41: /// </summary>

  42: public void ExecuteAsync()

  43: {

  44:     lock (this)

  45:     {

  46:         if (!_isBusy)

  47:         {

  48:             _isBusy = true;

  49:             ThreadPool.QueueUserWorkItem((WaitCallback)((state) =>

  50:             {

  51:                 int result = Execute();

  52:                 if (ExecuteCompleted != null)

  53:                 {

  54:                     ExecuteCompleted(this, EventArgs.Empty);

  55:                 }

  56:                 lock (this)

  57:                 {

  58:                     _isBusy = false;

  59:                 }

  60:             }));

  61:         }

  62:         else

  63:         {

  64:             throw new InvalidOperationException("Async operation in progress.");

  65:         }

  66:     }

  67: }

Using the Code

Using the code is fairly easy. Just create a new CommandLineTool object and call either the Execute() or ExecuteAsync() methods. Here is a snippet from one of the unit tests:

   1: string command = "cmd.exe";

   2: string arguments = "/c dir";

   3: CommandLineTool target = new CommandLineTool(command, arguments);

   4: int expected = 0;

   5: int actual;

   6: actual = target.Execute();

   7: Assert.AreEqual(expected, actual);

A more involved example (somewhat contrived) includes transformation of the results and asynchronous operations. [Results are transformed into FileObject objects].

   1: // The FileObject class ...

   2: internal class FileObject

   3: {

   4:     public DateTime ModifiedDate { get; set; }

   5:     public string Name { get; set; }

   6: }

   7:  

   8: // Code ...

   9:  

  10:     string command = "cmd.exe";

  11:     string arguments = "/c dir /s";

  12:     Func<string, object> transform = new Func<string, object>(s =>

  13:         {

  14:             DateTime modified = DateTime.MaxValue;

  15:             if (s.Length > 39 && DateTime.TryParse(s.Substring(0, 20), out modified))

  16:             {

  17:                 return new FileObject()

  18:                 {

  19:                     ModifiedDate = modified,

  20:                     Name = s.Substring(39)

  21:                 };

  22:             }

  23:             return null;

  24:         });

  25:     CommandLineTool target = new CommandLineTool(command, arguments);

  26:     target.ExecuteAsync(transform);

  27:     while (target.IsBusy) Thread.Sleep(0);

  28:  

  29:     List<FileObject> files = target.OutputObjects.Cast<FileObject>().ToList();

  30:  

  31:     ... etc ... 

Hope this comes in handy for someone. Let me know if you see any errors/problems or room for improvement.

Remember when a program was termed a “GUI Application” - a program with a Graphical User Interface? It had features like menus that “dropped-down” (or “popped up”), scroll bars to reveal more of the virtual surface area, a pointing device (still called a cursor), and lots and lots of Drag & Drop. Thankfully, that moniker has mostly died. Now it is time another term to fall by the wayside – the Web Application.

What is a Web Application?

Just what is a web application, anyway. An application delivered over the web? A Java applet, flash, or silverlight program running in a browser? How about HTML+Javascript with lots of Ajaxy asynchronous updating? What about plain old static HTML pages with hyperlinks to navigate between them?

Who are Web Developers?

Just who are these web developers? Are they PHP junkies, ASP.net gurus, Flex monkeys? Are the real web developers HTML+CSS+JavaScriptors? Rails developers? Web site designers? What about a C++ application that runs locally yet stores data in the cloud or communicates with other instances on the internet? Is the twitter client a web application?

The ubiquitous internet makes every application a web application. The term has lost its meaning, and the term must die.

Just like “GUI”. If an application requires a user interface, it will be a graphical one. If an application requires communication between computers, or presence on more than one computer, it will use the “Web”. In one way or another. Server side, client side, I don’t care. These days, every “application” is a “web application”.

Below are a few thoughts on the upcoming framework 4.0. I am not sure if the feature set is finalized, but I’ll comment on what I think are some of the more interesting features being planned for this release, as seen in the beta.

Managed Extensibility Framework

Plug-in architectures are a fairly common pattern for many scenarios, and I have seen quite a few over the years. MEF seems to be a nice combination of plug-in/IoC container/Dependency Injection framework. It’s use in Visual Studio 2010 should prove to give this framework a leg-up. I’m looking forward to using it as a replacement to the typical roll-your-own plug-in architecture. Not sure if it will be used where a complete dynamically composed extensible application is called for, however, that is exactly the scenario that MEF is designed for. I’ll see how it works out for plug-ins, first, before jumping in with both feet.

Language Enhancements

Some of these are obvious improvements: Optional arguments, COM interop/PIA enhancements. Some are welcome, such as named parameters. As for the new dynamic language features in C# ... well, I’m not qualified to comment on that yet. Interoperability with dynamic languages is a good thing. Whether or not I’ll be writing “dynamic c#” in the near future remains to be seen.

Base Class Library

I’ve blogged about one new class, the Memory Mapped File, in the recent past. There are a host of additional improvements, including the inclusion of the Task Parallel Library (TPL) that I believe came out of Microsoft research and has been release in CTP form a few times. Code Contracts look interesting, but I seriously doubt I will be adding pre-conditions, post-conditions and the like to very much code. I suppose there is a use for this – for some types of code; but I don’t see it for typical use. I had some experience with Eiffel in the past and did not care for it.

More Framework, Less P/Invoke

I would like to see more framework and less P/Invoke, particularly for new technologies such as those introduced in Windows Vista. There are managed wrappers provided for many things from Microsoft, but how about adding Windows 7 stuff to framework 4.0? It’s fine that wrapper libraries appear on CodePlex, but I would like to see the kind of treatment you would get from including new APIs in the framework – the way the System.Net.Sockets namespace is not simply a wrapper around the Win32 sockets API. I know, it’s hard – but what else is windows except a platform for development? If I was looking for merely an OS to run standard applications on, I could look elsewhere. Windows is first and foremost an application development platform for me. And for applications, I prefer managed code. Leave the C code (and COM goo) for drivers and low-level applications like virus scanners.

Visual Studio 2010

This looks like a big change. I’ll probably be using this for R&D (and also at home). But will probably wait about a year (or more) before converting all development projects over to vs2010. Although the beta seems pretty solid, my experience is that the first of a major change (vs2002/2003, the initial vs2008) in the Visual Studio line is not the greatest (contrast to vs6, vs2005 and now 2008/SP1). I love two things: multi-targeting and the ability to run different visual studio versions side x side. I expect that experience to be kept in vs2010. This allows developers to install the new tools with “no fear”, committing to them down the road. Goes a long way to increase adoptability. All that being said, the C++ team is saying “10 is the new 6”, and MSBuild can now be used for C++ code, etc., etc. I’ll keep an eye on thing in 2010.

F#?

Don’t know about this one. I’m thinking that the pile of technologies that I am now “expert” in is getting a little too large. However, I believe in the mantra “learn a new programming language every year”. It can be a bit confusing to switch from maintaining classic C code, early C++ code (mostly C with sprinkles of C++, MFC/ATL), later C++ code (STL, etc.), some VB6 and VBscript in classic ASP or office apps; then perform new development in JavaScript, C#, Windows Forms, WPF, Silverlight, and ASP.net. with a sprinkling of objective C for an iPhone app or two (quite a mix of frameworks and languages in that sentence, but you get the idea). I love it all. I’ll have to work F# in somewhere, because a functional language perspective brings new thought patterns, which is always a good thing. I suppose I’m glad it’s in the box, although at first thought I might want IronRuby or IronPython in there instead. I don’t see any production apps developed in F# in the next year or two, but an R&D project is likely.

-   -    -     -

I have to admit that I have not played all that much with the beta; been pretty busy over the past few months. But I’m looking forward to framework 4.0, and expect it will turn out to be a significant release.

There have been a few famous quotes on the subject – how much memory do you really need; how many GHz is enough; how much storage to hold all the books in the world; how many personal computers are enough for planet earth (let alone a single family), etc., etc. - I’ll stay away from predicting the future and just talk about what I need today.

2 THz processor

I need a 2THz processor. It took me 5 hours (300 minutes) to encode a family video to MP4 (H.264 720p) on a dual-core 2.66GHz machine. It is reasonable to want this to take just a few minutes (around the time it takes to burn an audio CD today) so that I could grab a video, throw some clips together, and encode the clips in an afternoon before dinner.

100 GB of RAM

Remember the days when large digital images could not fit in RAM? I would like my 5GB of video to fit in RAM – maybe two or three of them – comfortably – while editing the video. 100GB should be enough (for today). Speed up disk transfer, network, etc. by 100x or 1000x while we’re at it so we can push all this data around easier.

Petabyte Hard Drive

We need a place to store all these bits. A Petabyte would be a good starting point. While it is true that all of the emails, financial data, and other digital records I have can fit on a $20 8Gb flash card, and the music collections of everyone in the house combined add up to less than 100 Gb, we still have a cabinet full of DVD disks and nearly zero video digitally encoded and stored on the computers. Kind of like our audio data was some years ago.

Final Thoughts

Just sticking with what I need today – right now – I came up with these numbers. But I’m stuck with just a few GHz of cycles, a few Gb of ram, and a few Tb of storage. And I’ll be heating up the computer room for 5 hours at a clip with fans on high and CPU’s pegged at 100%.

We still have a long way to go.

As my home router’s “blocked web sites” list slowly began to fill up with open proxies (kproxy.com, getus.in, vtunnel.com, browse007.com, and anonymouse.org, etc.), I realized that it’s time to stop blocking and start monitoring.

DLink DIR-655 Event Logging

My DLink DIR-655 (great little router) contains an event logging feature that can send events, SYSLOG-style, over the network to a collector.

<110>Mon Aug 10 20:32:30 2009 DIR655 System Log: Web site geekswithblogs.net/ accessed from 192.168.1.91

<110>Mon Aug 10 20:32:30 2009 DIR655 System Log: Web site geekswithblogs.net/ScriptResource.axd?d=afn4d1BEk_dsfd45j-LusKDJDF accessed from 192.168.1.91

<110>Mon Aug 10 20:32:30 2009 DIR655 System Log: Web site geekswithblogs.net/Geekswithblogs.css accessed from 192.168.1.91

<110>Mon Aug 10 20:32:30 2009 DIR655 System Log: Web site geekswithblogs.net/WebResource.axd?d=PKDJF7383jdhksj_jd accessed from 192.168.1.91

<110>Mon Aug 10 20:32:30 2009 DIR655 System Log: Web site geekswithblogs.net/ScriptResource.axd?d=kalwo*hsdh%%jdjdgs90-48JSH accessed from 192.168.1.91

<110>Mon Aug 10 20:32:31 2009 DIR655 System Log: Web site geekswithblogs.net/?_TSM_HiddenField_=ctl00__mainContent_ScriptMana accessed from 192.168.1.91

<110>Mon Aug 10 20:32:31 2009 DIR655 System Log: Web site ads.geekswithblogs.net/a.aspx?ZoneID=1&Task=Get&IFR=False&PageID=51102& accessed from 192.168.1.91

<110>Mon Aug 10 20:32:31 2009 DIR655 System Log: Web site docs.geekswithblogs.net/DiscountASPNET/160x600_Champs_Trophy_6Mo_SM.gif accessed from 192.168.1.91

<110>Mon Aug 10 20:32:31 2009 DIR655 System Log: Web site www.gravatar.com/avatar.php?gravatar_id=4837367e8383a8b38376f accessed from 192.168.1.91

All I have to do is run a little server on the SYSLOG port, collecting messages as they come in and storing them in a database that can later be used as a source for reporting. I use the following regular expression to parse these messages:

private Regex _webSiteRegex = new Regex(@"^<\d{1,3}>(?<date>.+) DIR655 System Log: Web site (?<msg>.+) accessed from (?<source>[A-F0-9\.]+)", RegexOptions.Compiled);

This expression provides me with the date string, the partial URL of the site accessed, and the source of the request. The DLink router indicates the source as either an IP address or a MAC address. Wireless devices, such as an iPod Touch, are usually reported by MAC address while wired computers working off the router’s DHCP server are reported via IP. I decided to store MAC address information in the database, since the IP address for the various devices can change (I use DHCP). A simple ARP request can convert any of the IP address to MAC address on my subnet as needed.

To facilitate reporting, I decided to only store the hostname and domain portions of the URL in the database. My daily report, sent via email, displays the top 10 domains per device along with any domains that I have not seen before.

The Results

Well, the kids spend most of their time on facebook; I spend mine on various programming related sites; there are a fair amount of online shopping sites being hit, and of course there are way too many advertising sites.

Not that I didn’t already know this information.

But it will be nice to spot the occasional new domain that I might want to discuss with the kids. Sure beats the cat-and-mouse game of trying to keep the blocked list up to date, which I was beginning to lose.