Monday, July 30, 2007

Metrics: avoiding pitfalls

Karl E. Wiegers, who wrote a great book I've used called "Peer Reviews In Software", published these ten traps to avoid when implementing a metrics program. Anyone who has implemented a metrics program likely has experienced every one of these. It's a handy reference to put on your wall. Most are self explanatory and intuitive, once you see them.

The one I'm not quite sure I'm on board with is Trap #7: Using Metrics to Motivate, Rather than to Understand. Since an objective of measurement is to make data driven decisions, this is indeed, providing motivation to act. If the metrics show you spend a lot on rework, or it is trending up, then you and your team will be motivated to do something to improve. Conversely, if the metrics are trending down at a rate that is acceptable to you, you won't be motivated to do more. Maybe I am being picky here, in that you clearly don't want to punish with metrics (c.f. trap #6), and maybe "motivation" is along this same line. I'd be more in line with the trap if the word "Manipulate" replaced "Motivate", but maybe that's just plain obvious.

If we must have ten traps, I would replace #7 with "Automate everything" - a bold statement but once automated there is little to no long term cost, and the metric can be gathered even once it stops being useful - in case a change causes it to me meaningful once again.

No comments: