A big problem with depending on metrics is finding things to count that are actually useful measurements of whatever you’re tracking. A lot of bad management decisions can be traced to poorly chosen metrics. As a general rule, just because something can be measured doesn’t automatically mean that measurement will be useful. Tim Harford reviews a recent book on metrics:
Jerry Z Muller’s latest book is 220 pages long, not including the front matter. The average chapter is 10.18 pages long and contains 17.76 endnotes. There are four cover endorsements and the book weighs 421 grammes. These numbers tell us nothing, of course. If you want to understand the strengths and weaknesses of The Tyranny of Metrics you will need to read it — or trust the opinion of someone who has.
Professor Muller’s argument is that we keep forgetting this obvious point. Rather than rely on the informed judgment of people familiar with the situation, we gather meaningless numbers at great cost. We then use them to guide our actions, predictably causing unintended damage.
A famous example is the obsession, during the Vietnam war, with the “body count” metric embraced by US defence secretary Robert McNamara. The more of the enemy you kill, reasoned McNamara, the closer you are to winning. This was always a dubious idea, but the body count quickly became an informal metric for ranking units and handing out promotions, and was therefore often exaggerated. Counting bodies became a risky military objective in itself.
This episode symbolises the mindless, fruitless drive to count things. But it also shows us why metrics are so often used: McNamara was trying to understand and control a distant situation using the skills of a generalist, not a general. Muller shows that metrics are often used as a substitute for relevant experience, by managers with generic rather than specific expertise.
Muller does not claim that metrics are always useless, but that we expect too much from them as a tool of management. For example, if a group of doctors collect and analyse data on clinical outcomes, they are likely to learn something together. If bonuses and promotions are tied to the numbers, the exercise will teach nobody anything and may end up killing patients. Several studies have found evidence of cardiac surgeons refusing to operate on the sickest patients for fear of lowering their reported success rates.
