Metrics and Radiators

We all know that what gets measured gets done. So why is it that so many software organizations don't have data on their performance? My theory is that people are afraid to be measured. I have personally seen aversion to ascribing defect rates to individuals. The argument against it was that managers are not capable of interpreting the subtlety in the data, afraid that if one takes on harder work and incurs inherently higher defects rates, that managers won't know how to parse that out.

One of the better components of Agile Software Development is the information radiator. This is the prominent public display of data relevant to the project. I argue that a combined team and individual defect rate radiator will inherently drive the right behaviors in an organization. Management need take no additional action other than to keep the radiator current. Capable, normal Individuals and teams want to do better. They can rationalize why they might have worse metrics than other individuals or teams, but they will work to make it better if it's visible.

Other metrics I have worked on include those for efficiency and effectiveness of an organization. The first time you sit down with your team to define these, they probably won't get it. Mine came back and wanted to measure the number of meetings with agendas and number of meetings which finished on time. They chose those because meeting time seemed very wasteful to people. While interesting, this wasn't what I had in mind for measuring how well an organization is doing. So I went after another aspect of quality: delivering products on time.

One regular challenge to delivering products on time was dealing with change. Schedule changes from dependent components, requirements changes from customers or the market, design changes due to oversight,etc. What we agreed to measure was the number of contingency plans we had relative to the number we needed, and whether those contingency plans actually worked when needed. The other part related to that was how well we executed relative to plan. In order to do this you need a fairly granular plan. And a radiator. Also not a perfect measurement system, but one which should get the proper results.