Measurement is hard

I was going to title this post "ISO 15939", but then many of you would not be reading this. However, this last week I was introduced to a measurement world I am still trying to figure out. It looks compelling, and it is related to this standard.

One of our challenges is convincing people that they should expend the effort to measure something. One of the books on this subject I like is entitled "Practical Software Metrics for Project Management and Process Improvement", by Robert B Grady. In the book he describes how you use the "Goal, Question, Metric" paradigm to choose what to measure. Using this method you first state what it is you are trying to accomplish, then what questions you would ask to see if you are accomplishing that goal, and then decide how you would measure the answers. Very simple, in fact, practical. The book is full of lots of examples too, and it's dense, and to the point.

Fast forward to this week. I had the pleasure of listening to Rick Hefner, PhD, Director of Process Management at Northrup Grumman, talk about measurement and CMMI. He referred me to http://www.psmsc.com, which I'm still trying to wade through. It seems based on similar principles. PSM is a web site for Practical Software Metrics. The book by a similar title was written in 1992, and this organization has been meeting on a large scale since 1996; yet I had never heard of it; hopefully that's just me and this is old news to you guys. The book was written based on experience at Hewlett Packard. The PSM web site and group was sponsored from DOD and the Army (this may be why I haven't heard of it). But ... bear with me, it looks flexible, not DOD-ish.

I'm sure a number of you will react that we just need to use agile practices, work hard, and hire good people and forget measurement. I'm sorry, but I won't ever agree. You can't improve strategically without measurement, because you don't know what to improve, or if you actually did improve. If your product and work life is perfect, your customers are always happy, you never miss deadlines, you have more revenue and earnings than you can count, and you have fantastic work/life balance.. then fine, maybe you don't need measurement. If you have that environment, call me - I'll work for you. Otherwise I think you should check this out, just to know what it is and see if there are tools here for you. And by the way, eXtreme Programming advocates metrics - you have to measure the number of stories built in an iteration. So please don't tell me I don't get it.

There are a number of extensive white papers on the site, dealing with subjects such as how to measure your process improvement efforts, and measurement of security properties of systems.

They have a piece of free software (PSM Insight) that will help you with measurement that conforms to iso15939. I haven't run it yet, but was viewing their online demo and it looks useful. One could argue it's simply a graphical tool for your data, but it seems to also get you to conform your data in a way that uses industry standard metrics and indicators, so that others can understand you better. I would be interested in any comments from people who've used this tool.

Bottom line: check out http://www.psmsc.com