Paul C. Williams

Interfacing Technology & Business
View Paul Williams's profile on LinkedIn

Tuesday, December 3, 2013

Why are KPIs troublesome in professional environments?

There's no argument: Key Performance Indicators (KPIs) are useful. We, as busy people, must distill complex processes into easily understood and manageable chunks.  It's hoped that most KPIs, if improved, would actually impact the performance of our organizations. Why do we latch on to that so readily, and why doesn't it always work?

The Theory

It's a business school mantra that one cannot improve that which is not measured.  Scientific management needs to have easily gathered, analyzed and understood metrics in order to know where bottlenecks exist, and what parts of the process are ripe for optimization.  This is relatively easily done in some industries.  In manufacturing, you can measure process flow through each manufacturing line.  Similarly, calculating the number of units in various work-in-progress states is straight-forward.  It's very easy to see how decreasing process time improves process flow thus productivity and profitability.  Reduction of works-in-progress (which in manufacturing is equivalent to unproductive capital) directly affects the cash flow of the operation as well.

Both of these metrics can be used to compare the average values to specific measurements on specific lines to determine if a specific manufacturing line using a standardized process.  A manager can easily see that manufacturing line A has difficulties during the 2nd shift, because it's throughput is 20% lower than average. Similarly, we can look at line D to see what they're doing so much more effectively because their throughput is 20% higher than average.
Hypothesis Testing: Determine statistical significance
of a measurement against the known operational range.

Pay For Performance

It's natural then to implement a pay-for-performance scheme based on KPI measurements. In fact, KPI measurements can be used for a host of management processes that are functionally equivalent to pay for performance.  Consider an annual performance review that may use a KPI measurement as an objective component for the review. If the annual review might in the eyes of the reviewee be used as justification for promotion, termination, or even to which work space the employee gets assigned, it's important enough to the reviewee to ensure maximal results. The situation gets even more important for middle managers who may have variable compensation that depends on KPI targets.

KPI Development & Ownership

How are KPI targets developed?  The textbook answer to this question is to collaboratively decide using a cross-functional team of stakeholders from all levels to decide what metrics to use that accurately reflect improved support of the organizational strategy.  With a shared sense of ownership grounded in a firm understanding of the organization and strategy, all stakeholders have an incentive to make their metrics better.

The prerequisite here is that all the stakeholders really understand what the strategy is, and how their role supports the strategy.  Believe it or not, this understanding is exceedingly rare.

What seems to wind up happening is from somewhere on high, a directive is handed down that such-and-so will be measured, and we shall strive to improve this measurement.  In software development, the new measurement might be, in the most naive cases, lines of code, or bugs resolved.  In more enlightened organizations the metric might be estimated hours vs. actual hours for project execution.

In either case, those who are most subjected to the result of the analysis of the metric may not have much control over the outcome.  In the estimate vs. actual case, we might make the argument that the team ought to be good enough to estimate the effort required to execute the task. It's not uncommon though for teams to not know enough about external factors, such as specific integration requirements, or nonfunctional requirements like performance to accurately foresee all impacting tasks.

The net result is that contributors have a feeling of helplessness in affecting the KPIs they've been committed to.

A Perverse Incentive

What happens when contributors are impacted career-wise or financially by KPIs that were assigned to them by a management stack? They game the system. Teachers, held to KPIs encoded in law, cheat to improve their class test scores. I imagine the teachers and officials in these cases, already woefully underpaid, feel somewhat justified in their decision to cheat. Their KPI measurements are frequently impacted by factors outside their control. It's hard to get a kid to care about what I think most rational people can agree is a bogus standardized test. Kids can have impressively accurate BS detectors.

Kent Arnold, A former manager of mine once said about KPIs that "All software engineers are game theorists". By this he means software engineers and other professionals are savvy enough to figure out what is being measured and improve exactly that. Unless the metric actually supports an organizational goal directly, the KPI might encourage a behavior that doesn't support productive activity. Is 100% test completion a good target? What if a software developer were to, in the name of boosting the metric, decide to exclude tests that might not succeed? Perhaps the test suite is sandbagged with trivial tests to reach a 90% success measurement. In either of these cases, is the organization's goal of lowering maintenance costs and higher customer satisfaction achieved?

Are KPIs doomed?

This is a somewhat depressing line of thinking. Are metrics are so deeply flawed that they cannot be used in professional organizations?

I don't think so.  More importantly I think that the "right" way to implement KPIs will actually benefit morale and employee engagement while simultaneously providing better insight into the operations of the organization.

In an upcoming posting, I'll outline a model for generating useful KPIs in professional organizations.

No comments:

Post a Comment