Tim Kastelle has a nice piece on The Problem with Metrics.
[I]t pains me when I have to say that in business, our attempts at measurement are often ill-founded. Many times we end up using numbers as a defense against ambiguity and uncertainty. Most of the time, instead of trying to measure things, we might be better off just getting more comfortable with these states – because bad numbers are worse than no numbers.
Too often I find people report metrics on things simply because they can measure them, regardless of whether the metrics actually matter to the goals of the organization. Or, to Tim's point above, they report a number when they should be reporting a range. Or maybe the measures shouldn't be reported at all. After all, whatever you measure become a goal.
One of my colleague's favorite examples is from a food products manufacturer. Their goal as a company, of course, is to sell more of their refrigerated foods. But how was the manufacturing site measured? They were measured by the number of boxes shipped out of the plant. And when that number wasn't near its quarterly target, what happened? They hired a bunch of refrigerated trucks and shipped out a surprising number of EMPTY boxes. They hit their targets, didn't they?
The classic one in manufacturing is local efficiency. Each machine or work center should be running at maximum efficiency (meaning "up time" or products / hr or other measures). This usually drives each work center to run large batches because that makes them look very efficient - changeovers and setups are counted against their uptime metrics. The same measure drives the business to push additional work onto the shop floor to ensure everyone is "kept busy." Unfortunately, the real limiting factor (constraint) on the shop floor is going to be kept more than "busy," they will be insane with the amount of work waiting to be processed. And the poor work centers after the constraint will often be "starved" for work and will go hunt down other work. And when the jobs from the constraint finish, they sit there waiting for the make-work to complete. Don't forget about the metrics-be-damned expediting that happens when a customer gets on the horn with the president of the company: customer yells; expediting happens; metrics go in the dumper; people get dinged. The inevitable result is high work-in-process, long lead times, poor due date performance, poor quality, poor working relationships, and poor customer relationships.
What about in people or knowledge work environments? Surely it is harder to devise good metrics because the connection between knowledge work and the bottom line. This is true. However, one of the key assumptions about the manufacturing environment comes over into knowledge work: people must be kept busy, so we give them a lot of things to do. Surely, they will be able to prioritize and get the right things done and the right time. Sadly, no. Knowledge workers are subject to the same problems as any other part of the business: too many things happening, too many priorities, and too many projects demanding their attention.
Those are the problems. Solutions? Stop using these "local" measures of busyness and efficiency. Use measure that relate to the question, "How is what I'm doing supporting the goal of the business." For manufacturing, that is aligning operations with the constraint of the business - don't work on things that you aren't going to sell; don't work on more than the constraint can process. In knowledge work, it is similar: speed of completion should be the metric along with working on the right things.
There is much more to this story of metrics and measures, but the key is to measure the right things. And more importantly, stop measuring the wrong things - those measures will drive the wrong behaviors.
[Photo: "Measure" by Jeffrey]