There is a classic example of this in the book Jurassic Park. The island (Jurassic Park) had elaborate tracking systems to keep track of the dinosaurs. For efficiency purposes, the scientists set the systems to stop counting as soon as they reached the expected number of dinosaurs. The scientists believed that the genetically re-created dinosaurs on their island couldn’t reproduce. Therefore, they thought that they only needed to worry about decreases in number.
The story’s complication occurs when an outsider suggests that they increase the number at which the system stops counting. The scientists quickly find out that their assumptions about the dinosaurs’ reproductive abilities were wrong. The dinosaurs are multiplying.
Of course, that’s fiction. Yet, the very same problems occur in real life often with similarly unexpected and damaging results. The Jurassic Park scientists’ bias and assumptions tainted what and how they measured. Sound familiar? What you measure is what you get. That also means that what you measure is what you are looking to get. In his book, Why Smart Executives Fail, Sidney Finkelstein illustrates this problem with regard to Rubbermaid.
For a long period of time after the Great Depression, Rubbermaid was known for excellence in product innovation. By 1993, the company won Fortune Magazine’s “Most Admired Company” distinction. Yet, Rubbermaid crashed shortly after. Product design and innovation gave way to cost, availability, and efficiency. As the industry shifted, they did not. They got what they measured, but they were measuring (or at least focusing on) the wrong thing.
Advances in measurement have also fueled the problem. Techniques such as Lean and Six Sigma have given rise to a new appreciation of data and measurement. Changes in technology allow us to gather millions of data points daily and generate statistics to greater level of precision than ever imaginable. But all of this creates an illusion of understanding, certainty, and control. Six Sigma and Lean are excellent management tools. However, by themselves, they are no more effective at driving your business as a ledger or time and motion study. They are just tools.
The problem is that we now equate precision measurement with valid measurement. The result is that things that can’t be measured precisely are often not measured at all. This problem isn’t new. Paul Krugman, an economist and columnist for the New York Times, describes the problem in his paper, The Fall And Rise Of Development Economics.
“A friend of mine … has written a fascinating paper called "The evolution of European ignorance about Africa." The paper describes how European maps of the African continent evolved from the 15th to the 19th centuries...You might have supposed that the process would have been more or less linear: as European knowledge of the continent advanced, the maps would have shown both increasing accuracy and increasing levels of detail. But that's not what happened. In the 15th century, maps of Africa were, of course, quite inaccurate about distances, coastlines, and so on. They did, however, contain quite a lot of information about the interior, based essentially on second- or third-hand travelers' reports. Thus the maps showed Timbuktu, the River Niger, and so forth…Over time, the art of map making and the quality of information used to make maps got steadily better. The coastline of Africa was first explored…and by the 18th century that coastline was shown in a manner essentially indistinguishable from that of modern maps…On the other hand, the interior emptied out. …In a way, Europeans had become more ignorant about Africa than they had been before. …the improvement in the art of map making raised the standard for what was considered valid data. Second-hand reports of the form "six days south of the end of the desert you encounter a vast river flowing from east to west" were no longer something you would use to draw your map. Only features of the landscape that had been visited by reliable informants equipped with sextants and compasses now qualified.”In other words, Europeans became blind to those things that they couldn’t measure accurately. So instead of having a rough idea about the location of a river or town, analytical rigor caused them to show nothing at all. Does that make sense? Isn't a rough idea of the location and existence of river better than no knowledge of the river? You'd think so. Yet, I see this "if it can't be measured accurately, let's not measure at all" thinking all the time.
Leaders often dismiss measures that are subjective, qualitative, or anecdotal. Yet sometimes those measures can provide the best information. Ideally most of your measures should be statistically reliable and valid. However, a highly precise, reliable, and valid measure of the wrong thing cannot substitute for a rough measure of the right thing. Spending five minutes talking with your people will give you a much better sense for their level of engagement than any employee engagement study. Asking your customers if they are happy will yield much better information than detailed metrics about call center response time, product quality, or profit margin. Of course, those latter metrics are important too for the specific questions they answer. However, they don’t replace simple understanding.
Don’t get blinded by your measures. Use them to drive decisions and actions. But remember, metrics, in and of themselves, don’t provide value. They must be combined with your understanding of your business and current situation. Some decisions don’t require precision as much as they require understanding of the situation. Being told that it’s cold outside provides enough data know whether to put on a coat. Simple measures can be helpful.
---------------------
Brad Kolar is the President of Kolar Associates, a leadership consulting and workforce productivity consulting firm. He can be reached at brad.kolar@kolarassociates.com
