I think it's safe to say that most people, not just technologists or managers, but people in general, are “aware” of the failure rate of technology projects. I'm sure if you're reading this you've also read (or at least heard of) the now famous (notorious?) Chaos Report published by The Standish Group in 1994. If you haven't, let me summarize. Because that's what this post is all about - summarizing. According to most, it indicates that some 85% of technology (software) projects fail.
For years (I'm guessing since, oh, about 1994), the technology community have not only used this number, they've relied on it. It is the “reason” for contemporary development techniques. It is the “reason” for many books. It is the “reason” for many excuses (”Well, at least we're just as bad as everyone else!”). Even I fell prey to simply touting this one number instead of digging further into the research. In fact, The Standish Group should be disappointed after all of their hard work.
It turns out that the report is much, much more detailed than a simple failure rate. The Standish Group performed a very detailed analysis accounting for costs, schedules, feature sets, and quality factors. They interviewed small, medium, and large businesses across many industry verticals such as banking, retail, and health care. They went on to indicate three distinct types of project resolutions, not two. While it's true that they did summarize the information into “success/failure rates”, the study provides much more insight.
My point (and I'm getting to it) in all of this is that too often we, as technologists, look at our efforts as binary. That is, they either succeed or they fail. As a result, everyone else (and I do mean everyone), has the same perspective. In general, people look at a technology project simply as a success or failure without regard for the various aspects of the project. Even The Standish Group looked (and continues to look) at many variables when evaluating projects.
I'm not going to wax philosophical and say that “every project is a success if we learn from our mistakes.” That's just a tad too much of a cop-out for me. I am, however, going to suggest that we look at many factors associated with the project. I also think we should look outside the basic project management factors such as cost and schedule. These are somewhat “external” project factors. We should also consider internal factors like how well we are estimating, and how well we adhere to our own processes.
In addition to my opinion on this, iterative development principles suggest that this is “built in” to the definition. Instead of evaluating how well you are performing with regarding to cost, estimation, quality, schedule, process adherence, etc., etc. at the end of the project, you get to do it at the end of each iteration. What a great opportunity! Now every two-to-six weeks (for example) we can evaluate our success rate in various aspects. Additionally, we can make adjustments to our techniques thereby moving us closer to “success” in this or that aspects for the next iteration.
Iterative development allows us to view project successes and failures in discrete segments instead of en masse at the end. Combine this with a view that a project is not a discrete entity but is comprised of many different complementary facets. Together, these perspectives show that viewing a project as binary, as either successful or not, is harmful to our profession and (perhaps more importantly to us obsessive-compulsive types) simply inaccurate.
Stop hammering your team and yourself with a single “red light/green light”. Stop letting your boss hammer you! Give discrete information on each aspect of the project. And don't forget change management. Iterative development goes hand-in-hand with change management. A change management program will promote your success in various portions of the project. The next time you are asked “How's the project going?” give them the whole answer. Tell them about the budget, the schedule, the feature set, the resource changes, improvements in estimation, the reusable components, the lessons learned, the slip-ups, the overruns, the late nights. Don't focus on the so-called good or bad. Just focus on the facts and, whatever you do, do NOT give a single answer as to project status.