Measurement Blindness

There is an old saying that what you measure is what you get. While generally true, this statement can be a bit misleading. It doesn’t always work the other way around. What you’ve got isn’t always what you measure. In other words, there might be things falling below the radar simply because you aren’t measuring them.

There is a classic example of this in the book Jurassic Park. The island (Jurassic Park) had elaborate tracking systems to keep track of the dinosaurs. For efficiency purposes, the scientists set the systems to stop counting as soon as they reached the expected number of dinosaurs. The scientists believed that the genetically re-created dinosaurs on their island couldn’t reproduce. Therefore, they thought that they only needed to worry about decreases in number.

The story’s complication occurs when an outsider suggests that they increase the number at which the system stops counting. The scientists quickly find out that their assumptions about the dinosaurs’ reproductive abilities were wrong. The dinosaurs are multiplying.

Of course, that’s fiction. Yet, the very same problems occur in real life often with similarly unexpected and damaging results. The Jurassic Park scientists’ bias and assumptions tainted what and how they measured. Sound familiar? What you measure is what you get. That also means that what you measure is what you are looking to get. In his book, Why Smart Executives Fail, Sidney Finkelstein illustrates this problem with regard to Rubbermaid.

For a long period of time after the Great Depression, Rubbermaid was known for excellence in product innovation. By 1993, the company won Fortune Magazine’s “Most Admired Company” distinction. Yet, Rubbermaid crashed shortly after. Product design and innovation gave way to cost, availability, and efficiency. As the industry shifted, they did not. They got what they measured, but they were measuring (or at least focusing on) the wrong thing.

Advances in measurement have also fueled the problem. Techniques such as Lean and Six Sigma have given rise to a new appreciation of data and measurement. Changes in technology allow us to gather millions of data points daily and generate statistics to greater level of precision than ever imaginable. But all of this creates an illusion of understanding, certainty, and control. Six Sigma and Lean are excellent management tools. However, by themselves, they are no more effective at driving your business as a ledger or time and motion study. They are just tools.

The problem is that we now equate precision measurement with valid measurement. The result is that things that can’t be measured precisely are often not measured at all. This problem isn’t new. Paul Krugman, an economist and columnist for the New York Times, describes the problem in his paper, The Fall And Rise Of Development Economics.

“A friend of mine … has written a fascinating paper called "The evolution of European ignorance about Africa." The paper describes how European maps of the African continent evolved from the 15th to the 19th centuries...You might have supposed that the process would have been more or less linear: as European knowledge of the continent advanced, the maps would have shown both increasing accuracy and increasing levels of detail. But that's not what happened. In the 15th century, maps of Africa were, of course, quite inaccurate about distances, coastlines, and so on. They did, however, contain quite a lot of information about the interior, based essentially on second- or third-hand travelers' reports. Thus the maps showed Timbuktu, the River Niger, and so forth…Over time, the art of map making and the quality of information used to make maps got steadily better. The coastline of Africa was first explored…and by the 18th century that coastline was shown in a manner essentially indistinguishable from that of modern maps…On the other hand, the interior emptied out. …In a way, Europeans had become more ignorant about Africa than they had been before. …the improvement in the art of map making raised the standard for what was considered valid data. Second-hand reports of the form "six days south of the end of the desert you encounter a vast river flowing from east to west" were no longer something you would use to draw your map. Only features of the landscape that had been visited by reliable informants equipped with sextants and compasses now qualified.”

In other words, Europeans became blind to those things that they couldn’t measure accurately. So instead of having a rough idea about the location of a river or town, analytical rigor caused them to show nothing at all. Does that make sense?  Isn't a rough idea of the location and existence of river better than no knowledge of the river? You'd think so.  Yet, I see this "if it can't be measured accurately, let's not measure at all" thinking all the time.

Leaders often dismiss measures that are subjective, qualitative, or anecdotal. Yet sometimes those measures can provide the best information. Ideally most of your measures should be statistically reliable and valid. However, a highly precise, reliable, and valid measure of the wrong thing cannot substitute for a rough measure of the right thing. Spending five minutes talking with your people will give you a much better sense for their level of engagement than any employee engagement study. Asking your customers if they are happy will yield much better information than detailed metrics about call center response time, product quality, or profit margin. Of course, those latter metrics are important too for the specific questions they answer. However, they don’t replace simple understanding.

Don’t get blinded by your measures. Use them to drive decisions and actions. But remember, metrics, in and of themselves, don’t provide value. They must be combined with your understanding of your business and current situation. Some decisions don’t require precision as much as they require understanding of the situation. Being told that it’s cold outside provides enough data know whether to put on a coat.  Simple measures can be helpful.

---------------------
Brad Kolar is the President of Kolar Associates, a leadership consulting and workforce productivity consulting firm. He can be reached at brad.kolar@kolarassociates.com

Outcomes and over-acting

This past weekend I had the arduous task of taking apart a wooden play set in my yard. Dismantling the boards and pulling hundreds of nails got me thinking about effort, activity, and outcomes.

At one point, I had to remove the swing hangers. The hangers are long bolts that run through a cross beam and are secured at the top by a nut. There is a metal cover over each of the hangers. Four nails secure the cover. To remove the hanger, you must first remove the cover to access the nut.

As I was prying up the cover, I realized something. I didn’t have to remove all four nails. I simply had to remove two nails and bend it back. My outcome wasn’t to remove the cover - that was an interim step. My outcome was to expose the nut in order to remove it (actually, my outcome was to remove the hanger – removing the nut was the most efficient way to do that).

Removing all four nails would have been over-acting relative to my outcome. In this case, over-acting had little cost other than maybe some nicked fingers and sore muscles. In a work setting over acting can cost time or money - resources that are a lot scarcer these days.

It made me wonder how often we over-act. I see it happen a lot. We lose sight of the outcome and continue to focus on executing the activities associated with it.

Misguided metrics are a major source of over-acting. When metrics are focused solely on activity, it’s no surprise that they generate a lot of activity. Without over-arching outcome measures, activity metrics can easily lead down a path of perpetual motion with little progress.

Bottom-up reporting, diagnosing, and communicating are a second major source of over-acting. Bottom-up means starting with the activity metrics and working your way to the outcome measures. This type of approach probably results from our being used to thinking about our world in a linear, cause-effect manner. We tend to measure, report, and communicate what is happening in the order that it happens. That’s not effective from a management point of view. The first and foremost area of focus should be the outcome. That drives your actions.

Lack of clarity is the third major source of over-acting. Too often, leaders are at a loss to define the end game clearly. Instead, they take the easier route. They define activities that can easily be measured and reported. Sometimes organizations bring me in when they believe they are having a performance problem. My first question is always the same – what outcomes are not being met. I’m amazed at the blank looks I get when I ask that question. The response is usually, “Can’t you just look around and see what is and is not working?” The answer is no. I can look around, that’s easy. But, I can’t tell what is or isn’t working. If the organization is shooting for 4.5 out of 5 on customer satisfaction, then its processes are working if satisfaction is at 4.6. They are broken if satisfaction is at 4.3. Without knowing the end game, the exercise in finding problems also becomes an activity. All people and processes CAN be improved. The question is whether they SHOULD be improved. You can only answer that by understanding what you are trying to achieve.

Take a look around. Talk to your people. Find out how they determine when enough is enough. Is it the completion of an activity or the achievement of an outcome?

---------------------------
Brad Kolar is the President of Kolar Associates, a leadership consulting and workforce productivity consulting firm. He can be reached at brad.kolar@kolarassociates.com.

Stop being a mystery writer. Become a prosecutor instead.

Nothing is more exciting than plowing through a three hundred-page mystery novel waiting with anticipation to find out “who did it”. Unfortunately, that excitement doesn’t translate well to non-fiction. Most executives want answers, not long drawn out sagas. Yet, we often make them wait.

It’s time that we shift our focus. Throw away the detective cap and pipe and put on your pinstriped suit. It’s time to become a prosecutor.

Think about the way a prosecutor lays out the opening argument for his or her case. What’s the first thing they say?

“On June 5, 2010 Mr. Gray entered the music room, picked up a piece of conduit and struck and killed Ms. Goldfinch.” The prosecutor then lays out the case. Story complete. The rest of the trial is where the evidence is presented to support that initial assertion.

But what if prosecutors adopted the way that many of us present information? The opening argument might go something like this.

“On June 5, 2010, Ms. Goldfinch returned home from a day at the track. While reading in her music room, she was beaten with a piece of conduit. Ms. Amber who came by to pick up Ms. Goldfinch for a dinner engagement called the police at 5:32. The police ran their investigation.

They found that the killer entered through the backdoor. The police considered three suspects, Mr. Gray, Col. Relish and Professor Peach. The police followed their standard investigative procedure. They dusted for fingerprints and collected DNA evidence from hair follicles left at the scene.

They then interviewed each of the suspects. Mr. Gray said that he was on a hunting trip during the crime. Professor Peach and Col. Relish said they were together at an awards banquet. The police spoke with one hundred people at the banquet. Forty-five were women and fifty-five were men. Of the men, thirty were between eighteen and thirty-five years of age while the others were all older than thirty-five. Ninety-five percent of the witnesses recalled seeing Professor Peach and Col. Relish. The police then interviewed the individuals with whom Mr. Gray had gone hunting. There were ten in total. Five lived in the city, four in the country, and one wasn't sure. Three of the men were over six feet tall. None of the men could corroborate Mr. Gray’s story.

The police then compared the DNA and fingerprint evidence to samples taken from the three suspects. There was a one-point match against Professor Peach, a three-point match against Col. Relish, and a five-point match against Mr. Gray. The DNA did not match Professor Peach or Col. Relish although our forensic scientists made an interesting discovery. Col. Relish is a carrier of a very rare form of Hepatitis C. Mr. Gray's DNA matched the sample found at the crime scene.

The police also interviewed Ms. Amber regarding each of the men’s relationship with Ms. Goldfinch. Ms. Amber reported that Col. Relish had never met Ms. Goldfinch. She said that Ms. Goldfinch and Prof. Peach had been dating for several years and rumor had it that he was planning to propose to her. She also said that Mr. Gray was Ms. Goldfinch’s college sweetheart who she dumped for a circus clown. She said that on several occasions she heard Mr. Gray swear to get even and ensure that if he couldn’t have her, no one would.

Given that Mr. Gray’s DNA was found at the site, that his alibi didn’t check out, and that he made threats against Ms. Goldfinch, we believe the he is guilty.  Oh, by the way, he's also an electrician and has access to conduit.”

That seems a lot less effective doesn’t it? Putting the answer at the end is great for mystery novels. It doesn’t work for business decisions and actions. Lead with the answer and then make your case. Your audience will let you know what additional information they need. It will also help focus your discussion on the decision or action rather than the minutia of the process.

--------------------------
Brad Kolar is the President of Kolar Associates, a leadership consulting and workforce productivity consulting firm. He can be reached at brad.kolar@kolarassociates.com

Do your performance management processes leave your best leaders on the cutting room floor?

In their March, 2001 Harvard Business review article “The New Path to the C-Suite”, Boris Groysberg, L. Kevin Kelly, and Bryan MacDonald state that

“Once people reach the C-Suite, technical and functional expertise matters less than leadership skills and a strong grasp of business fundamentals”

Their article goes on to describe the real abilities needed across the spectrum of C-suite executives (CIO, CMO, CFO, CHrO, etc.). Few have to do with the basic functional and technical requirements of their respective disciplines.

Their article made me question current performance management systems. How well are they identifying the people with the best leadership potential?

In many organizations, leadership ability isn’t measured until a person becomes a leader. Prior to that, performance measurement is often focused on detailed competency and performance models associated with functional or technical skills. That means that organizations aren’t moving people into leadership positions based on their ability to lead. This would explain the current leadership crisis that many organization face.

This isn’t to say that among the functional and technical experts there aren’t great leaders. However, it does pose a problem. Those who are just “good” functionally and technically but show great leadership promise might be getting left behind. When was the last time your organization moved an “average” performer into a leadership position? Yet, how often do “top” performers move into leadership and flounder? If your organization defines good, average, and top solely on functional and technical competence, it probably happens a lot.

Perhaps it is time to rethink our performance management and succession planning processes. Leadership ability should be measured and tracked from day one, regardless of level. We need to break down the perceptions that our top leaders come from a subset of our top experts. Your best leaders might be the ones who know how to draw the greatest contribution from those experts.

-------
Brad Kolar is the President of Kolar Associates, a leadership consulting and workforce productivity consulting firm. He can be reached at brad.kolar@kolarassociates.com.