Geekus Con Livus

Malcolm Anderson's home for Geeks With Lives
posts - 91 , comments - 79 , trackbacks - 23

The only metric with any value

There's a lot of talk in the Scrum world about metrics. What's the velocity? How big is a story point?  How many story points is that team producing per man hour?

 

People are sadly missing the whole point.  Take your measurements up a level or two.  When you get down to it, the only metric that makes any difference, is ROI.

 

The problem is that often times, the developers work in a dark hole, far removed from the realities of how exactly they get paid.  A bigger problem is that mid-level managers tend to be further removed from the realities of ROI.  A lot of times mid-level managers get tasked with tracking their teams "productivity" using things like, "lines of code", or "completeness of the productivity reports."

 

Monetize your projects and then track your velocity against business value (real dollars). 

 

When your development teams can say, "Last year, our team cost the business 2 million dollars and we know that because of our efforts, the company saved 2 million dollars in waste and increased revenues by another 4 million dollars." At that point you have just moved your development team from a cost center, to a profit center.  You might have to give them a raise, but they have demonstrated that they have earned it.

Print | posted on Monday, April 19, 2010 12:20 PM | Filed Under [ Agile Development Scrum ]

Feedback

Gravatar

# re: The only metric with any value

ROI is important. It doesn't matter whether its a metric or not. However it's something you should measure and it is the ultimate measure of improving processes and the way you do things.

However I think you are missing the point of metrics. Fundamentally you need to understand why you are measuring something. You should use a metric to make a decision, and you need the right metrics to make good decisions. If you aren't using metrics to make any decisions then don't measure it.

so, "what features should we do over the next month?" well, what ones do we think will give best ROI is a good indicator.

"how many of them should we do? well, we made $1000000 ROI... "so ummm, how many should we do?" For planning, you need other metrics which helps make planning decisions. Something that shows how much you can get done and how good you can estimate what needs to be done.


If you ask the question how well is Product Team X doing? then saying 200 story points per 3 months is not as good as saying $1000000 for a quarter.

ROI is important, but it alone cannot provide all the information you need to make good decisions at all the levels of the development process.

its why its good to have a number of KPIs. ROI wold be one. You'll want to measure quality of the product also.

Personally I think if you have no metrics, then ROI should be the first metric you find out.

A lot of this is covered in lean software development / manufacturing.
4/19/2010 6:56 PM | Keith Nicholas
Gravatar

# re: The only metric with any value

I agree completely with Keith. ROI should not be your only metric. It should be part of decision process, but not the only metric.

Let's say the VP of sales comes to you and says we have 3 projects for you to work on this year, and you do all the work and it you end up with something like this:

Project A - Increase Revenue 250k/quarter, 300 Story points
Project B - Increase Revenue 175k/quarter, 150 Story points
Project C - Increase Revenue 100k/quarter, 50 Story points

How do you decide which one projects to work on? Obviously you want to generate the most revenue possible so you pick Project A right? That's 1million a year. But maybe that's not the best decision.

What if you have a metric that states you can do 50 points per quarter? Then the decision changes. Now you can do both Project B and C in year, with a increase of 275k/quarter in revenues. Which is another 100k/year. That's 10% better. Actually it's even more as it would take you 18 months to complete Project A. During the 6 months extra it takes you complete project A, Project B & C have just brought in another 550k.

While at the end of the day, all the executives care about is ROI/Revenues/Cost Savings, you NEED the metrics to be able to maximize the ROI and make intelligent decisions.
4/19/2010 8:05 PM | thorkia
Gravatar

# re: The only metric with any value

Thorkia

I think the details you are bringing up are important.

ROI is a generated metric. You would HAVE to have confidence in the accuracy of the the following numbers.

What is your fully loaded developer cost?
What is your teams velocity?
How big is your project in story points?
How much revenue will this project generate?

All of these are going to be soft numbers that you are not going to want to look at too closely (the cost of more analysis will not yield more precision, just the illusion of precision, I call that waste).

To get back to your scenario:

If all things are equal then Project C would be the best choice to start with. The 300K that your company will generate for the rest of the year will help offset the cost of supporting the development team.

Follow Project C with Project B, you go into Year 2 with a million dollar a year income generated by ITs efforts. Now, as a company, you are in a place to have the development of Project A paid for by the previously generated revenues.

A possible downside of this is that your developers will be a little more cocky than they already were.
4/24/2010 4:22 PM | Malcolm Anderson
Post A Comment
Title:
Name:
Email:
Comment:
Verification:
 

Powered by: