Do you know your metrics? The three question test

How well do you know your metrics? Chances are you know the current values of them (if you don’t, that’s an issue). But, do you really understand what that number is telling you? Here are three simple questions that you should be able to answer for any metric:

1. In plain language, what does this metric tell you?
2. What’s its equation?
3. Is it better for the metric’s value to be a high number or a low number?

In plain language, what does this metric tell you?
Can a company that has always been profitable and continues to be profitable go bankrupt (assuming business remains consistent and there are no unexpected surprises or disasters)? About 80-90% of the leaders to whom I ask this question get it wrong. They say “No, if they are making money they shouldn’t go bankrupt.” As many small businesses learn very quickly, profitability doesn’t mean that you have cash. It just means that the revenues on your books exceed the costs. Cash flow is a measure of how much money is coming into (and going out of) your organization.  It is important to understand what a metric is (and is not) telling you.

Sometimes we make assumptions based on the name of the metric. For example, one organization had a “customer satisfaction” metric. Despite its name, it didn’t directly measure customer satisfaction. It measured the organization’s percentile ranking on customer satisfaction. Percentile rankings are relative. If every other organization is great and their organization was good, their percentile ranking would be low. If, on the other hand, every organization was horrible and their organization was just bad, the percentile would have been high. Because they were missing their target, many leaders said that their customer satisfaction wasn’t good. However, they struggled to reconcile this with their actual interactions with customers as many seemed quite pleased. The problem was that they were in a highly customer-centric industry and competition was high.

This metric didn’t measure satisfaction. It measured the number of better choices that customers had when determining where to take their business.

If you can’t explain your metrics in plain, simple language, you might not really understand them. Plain, simple language means that you can explain them without just rehashing their formula. You should be able to tell someone what the metric means to you and your business.

What’s the equation?
Can you write out and explain the formula for your metrics? Do you understand each component? Do you know if the numbers used in the calculation (or the output) are actual values, projected values, planned values, or annualized values? That makes a big difference as well.

Do you know the range of possible values that your metric can take? For example, some organizations like to re-scale survey scores in order to widen the results. One company multiplied satisfaction scores by twenty to convert them from a five point scale to a one hundred-point scale. About 80% of the leaders with whom I spoke though that the final scale was zero to one hundred. That was incorrect - it was twenty to one hundred (since the survey started at one not zero). That makes big difference when your scores are in the 60-80 range.

Most leaders also overstated the impact of changes to that new score. A five-point difference in the final score sounded good until it was converted back to original scale. 
Do you understand why the formula is written in a certain way? For example, profitability can be computed using either subtraction or division. But division provides more information as it helps you understand the rate of profitability as opposed to just the absolute amount.

Finally, do you know if your metric is a relative or absolute number? More importantly, if it’s relative, do you know what the metric might NOT be telling you? In the earlier example, the relative customer satisfaction metric masked the actual level of customer satisfaction.

Is it better for the metric’s value to be a high number or a low number?
This one might sound obvious, but some metrics are tricky, especially when they are mixed on a scorecard. I’ve seen cases a metric was lower than its associated targets. The leaders interpreted this as missing the target (since all of their other metrics were designed so that higher numbers were better). Their corrective actions continued to lower the value of the metric. They couldn’t figure out why they weren't “fixing” the problem (when in reality they were over-fixing it). They wasted a lot of unnecessary resources over-optimizing the metric.

Do you understand what levers you can pull to make influence the metric? Do you understand the relationship between positive or negative changes to one of the metric’s components and the metric itself (e.g., if a component goes down, which way does the metric go?)

Every day, leaders use metrics to make decisions and drive actions. However, they often “fly blind” when it comes to actually understanding the metric. If you can’t answer all of these questions for the metrics you use, there is a good chance that you aren’t making the best decisions.

-------------------------------
Brad Kolar is the President of Kolar Associates, a leadership consulting and workforce productivity consulting firm. He can be reached at brad.kolar@kolarassociates.com.

Data doesn’t tell you WHAT to do just WHERE to do it

What should you do with unprofitable customers? What about poor performing employees? Would you allow a quality problem to continue? If your sales were steadily dropping in one market, what actions would you take?

Even though I didn’t give you any data, you probably know the answers to these questions. What you don’t know is which customers are not profitable, which employees have performance issues, where the quality problems are occurring or which markets are losing sales. For those questions, you need data.

Leaders often tell me that they cannot make a decision until they see the data. While it’s true that you can’t complete the decision without the data, you can get pretty far down the path.

In fact, you’ll probably make better decisions if you think them through prior to seeing the data. For example, consider rating employee performance. Ideally, you should have a set of criteria in mind for what high, average, and low performance look like. Then it’s simply a matter of comparing individual performance against those criteria to get the result. For example you might get something that looks like this:

However, sometimes that’s not what happens. Perhaps we look at the data and see that our up-an-coming superstar, Jane, is third on the list. We “know” that Jane is a high performer. She just had a tougher assignment than Bill or Karen. Certainly if they ran into the challenges that Jane faced, they wouldn’t have done so well. We shouldn’t penalize Jane because we give her the hard tasks, right? All of this might or might not be true, but it’s the conversation that goes on in our head. We decide that our high performer cut-off should come just after Jane, not after Karen as our original criteria might have suggested.

Then, we see Bob. Poor Bob. He really screwed up a couple of years ago and we haven’t forgotten. Sure, he did a pretty good job. Unlike Jane who caught a bad break, he probably just got lucky. We decide to put the low performer cut off just above him. After all, how can we consider a poor performer like Bob to be “average?” The result is a very different picture of your workforce.

Of course, these kinds of decisions are much more subtle. Often they happen unconsciously as our bias takes over the way we view the data and make our decisions. It does happen and not just with performance decisions.

In his book, “The Logic of Failure: Recognizing and Avoiding Error in Complex Situations”, Dietrich Dörner describes similar errors that contributed to the Chernobyl disaster. The team running the reactor (who turned out to be a group of expert scientists) dismissed the data, alarms, and other warnings. Dörner concluded that:

“The Ukrainian reactor operators were an experienced team of highly respected experts who had just won an award for keeping their reactor on the grid for long periods of uninterrupted service. The great self-confidence of this team was doubtless a contributing factor in the accident. They were no longer operating the reactor analytically but rather ‘intuitively.’ They thought they knew what they were dealing with, and they probably also thought themselves beyond the ‘ridiculous’ safety rules devised for tyro reactor operators, not for a team of experienced professionals.”

The people who devised the safety rules based them on a set of criteria based in science. However, when the scientists filtered the real-time data through their personal experience and biases, they decided to move the lines that delineated when corrective action was needed. The result was one of the worst disasters in history.

I’m not suggesting that we completely ignore our experience. However, there are ways to use that experience constructively. Proactively figuring out what actions you’ll take and the criteria for taking those actions reduces the chance of being influenced by your bias. There will be times when, after seeing the data, you might choose to make an exception. However, in that case, at least you are making the exception consciously.

If you are waiting for your data to tell you what to do, you'll be waiting for a very long time.  Your data doesn't "know" about your business, it's just a bunch of facts.  What to do is based on your knowledge, experience, and judgment.  Your data can only tell you where and when to do it.

--------------------------
Brad Kolar is the President of Kolar Associates, a leadership consulting and workforce productivity consulting firm. He can be reached at brad.kolar@kolarassociates.com.

Even the most transactional activities have outcomes

I’m often asked at what point leaders can stop focusing on outcomes. The argument I hear is that some people's jobs, especially those lowest in the organization, are just about executing tasks.

I disagree. Even the most transactional activities have outcomes – otherwise why bother doing them?

Every task, no matter how small, is done for a purpose. A call center operator’s job isn’t just to complete a call. That transaction should achieve something – it needs to satisfy the customer’s need in a way that is efficient and cost effective to your business. Ideally it should make the customer feel better about your business than before they called. Those are outcomes. How many times have you called a support line for help and hung up feeling no better off (or even worse off) than when you started?

When you don’t explicitly state an outcome your people don’t have clarity on what they are trying to achieve. As a result, they use completion of the task as the measure of success.

These “micro” outcomes are what make or break your business. After all, your “major” outcomes are all built upon the success of each individual transaction and interaction.

If you cannot think of an appropriate outcome for a task or deliverable, stop and take a step back. You might not fully understand why that task is being done or why that deliverable is being created in the first place.

And if you don’t know, neither do your people.

------------------------
Brad Kolar is the President of Kolar Associates, a leadership consulting and workforce productivity consulting firm. He can be reached at brad.kolar@kolarassociates.com.

Want results? Keep two sets of books (legally)!

We often hear stories of companies that keep two sets of books. One set is used for the government or other regulatory agencies. This one paints of picture of the business that falls within the governing body’s accepted ways of doing business. The other set actually shows what’s going on in the business. That’s the one from which decisions and actions are made. Of course, keeping two sets of books in this way is illegal and not a good idea.

However, lately I’ve been recommending this strategy to the leaders with whom I work. Well, I’m not recommending this exact strategy - there is a slight twist.

Many organizations fall short when it comes to goal settings. Attempts to have “objective” or “quantifiable” goals often lead to statements that are tactical and activity-based. After all, it is much easier to see and prove that you’ve implemented a new process than it is to show that you are actually delivering better results due to that process.

Leaders who attempt to create goals that are truly results oriented fall into two traps:

1) They might not be able to provide an exact metric (or an “acceptable” metric) and are criticized and forced to turn them into activity-based goals (which would have been easier to do in the first place)

2) Their peers all have activity-based goals which are easier to achieve. This puts the leader at a disadvantage at the end of the year when his or her achievements are compared against peers.

The solution - two sets of “books”; the goals that you create for the formal HR process and the goals that you create for yourself to run the business.

The first set is written for your boss, HR, or whoever else is involved in your Performance Management. These goals comply with the standards, style, guidelines, and expectations set forth by the formal process. By aligning with the process, you’ll ensure that you won’t be penalized for trying to achieve real results.

The second set should be the outcome-based changes that you are trying to affect. Your personal measure of success and all of your actions should be driven by achieving these goals. You shouldn’t be satisfied until they are met, regardless of the status of your formal goals.

By staying focused on true outcomes, you’ll achieve three things:

1) You’ll increase your chances of actually impacting the business

2) You’ll still achieve the formal, activity-based goals (assuming that you have to do the activities to get the outcomes). However, the opposite isn’t always true - achieving the activity doesn’t always guarantee the outcome.

3) You’ll set yourself up for long term success. Sure, people who hit their annual activity-based goals win in the short term. However, over time, the people who succeed are the ones who become known for having a real impact on the business. Those short-term, activity-based contributions are soon forgotten along with the person who achieved them.

In an ideal world, there wouldn’t be a need for two sets of goals. Unfortunately, few of us work in an ideal world. Until organizations and leaders truly embrace an outcome focus, you might have to keep a second set of books.

The good news is that, in this case, the second set isn’t only legal, it actually benefits both you and the organization.

--------------------
Brad Kolar is the President of Kolar Associates, a leadership consulting and workforce productivity consulting firm. He can be reached at brad.kolar@kolarassociates.com.