The Dangers of Counting

On Slow Leadership today they’re talking about Occam’s Razor. Choosing the simplest explanation. And the dangers of setting numerical targets.

But metrics are so easy to collect these days. The temptation to count things is great.

Let’s say you are deploying a new process, or tracking progress on an existing one. What harm could there be in counting, looking at the numbers, and making decisions based on them? Isn’t that sound management?

The problem is assuming that by counting something you learn more about it.

Actually all you learn is how many of something there were.

You don’t know anything about those somethings, nor why the count came out as it did.

When you count deployment events, there is no positive integer count that tells you much by itself.

Even zero doesn’t tell you unequivocally what happened. I can think of at least 3 possibilities about the events:

a) Didn’t happen
b) Happened but insubordination or fear among the counters
c) Happened but misunderstanding about what to count

Things get worse if the numbers are positive. How do you know whether a number is large or small? More, or less?

For example, trying to measure software development, let’s say you counted 24 high-severity defects last month. Is that high or low? Is it more or less than the month before? Kind of depends, doesn’t it, on how much code you wrote, how much testing was done, how good the testing was, which defects were recorded, and so on.

Forget distributions — pie charts and the like. How do you know if people understood which category an event belonged in?

What does all this leave us with? No graphs? No counting? No decision-making based on numbers? Back to the Stone Age?

No. It just means something counter-intuitive in today’s modern world of computers: knowledge before counting.

First learn the qualitative details of your events. What is a satisfactory defect entry? A relevant test result? A productive code review? Make sure all participants can recognize them.

Then you can start counting.

1, 2, 3 …

P.S. Want extra credit? Read just the first page of What is Mathematics? by Courant and Robbins. The section is called “The Natural Numbers: Introduction”.

Advertisement

Monitoring for Encouragement

When you deploy a new process, one question that comes up is, “Will it last?” And the usual solution is monitoring — recordkeeping and reporting to track how it’s going. But do we really want to check up on everyone to make sure it’s really happening, and going smoothly, and producing results? Or better to just let people alone so they don’t feel that somebody’s watching them?

The concern stems from people’s thinking of monitoring as a negative thing. If monitoring shows problems, will their managers criticize them? If monitoring shows fewer results than expected, will people think the process isn’t working?

But monitoring can be a positive tool if used correctly. Promise yourselves that:

• If monitoring shows the new activity not happening enough, managers will take that as a signal to ask their people what they need help with. A manager’s job is to provide what his people need to do their jobs — perhaps something needed to make it happen is missing. Ask and find out.
• If monitoring shows the activity not producing enough results, wait! Focus first on the results that are there. Seek out the people and groups that produced them, and compliment them. When people are encouraged and rewarded (even just with positive attention), they’ll do more, as well as feel more comfortable asking for help if they need it.

While we’re at it, monitoring does not have to be complicated. Here’s an example, in this case, for code reviews. Something everyone believes in, but can be hard to keep going.

Make yourself a spreadsheet with just four columns: Scope of Planned Review, Planned Review Date, Did it Happen? and Number of Issues Found.

For the first good while, all you have to do is make a check mark for each planned code review that actually happened, and find out how many issues were found. Yes, there’s a lot more to code review than these two basic measurements. But as a tool for help and encouragement, they’re all you need.