Friday, June 23, 2017

Metrics Anecdote

Here is a short, very simple fictional (adapted from reality) anecdote about metrics in the realm of knowledge management.

"Barbara, how many workshops have we done in the past 8 years?"
"Let me check.... 25."
"We've done 25 workshops in the past 8 years?"
"Yes."

And the number 25, whatever it means, is now a data point for someone in management who needed to know how many workshops we have implemented.  Someone needed numbers.  To say "we conduct knowledge sharing workshops" isn't the same as "we have conducted 25 knowledge sharing workshops in the past 8 years."  And yet, that number is meaningless.  Here's why.

"Barbara, how many workshops have we done in the past 8 years?"
"Probably more than 20. Why do you ask?"
"I need to document our activities with some quantitative metrics."
"Oh, then the total number of workshops isn't that relevant or useful because we used to do full day workshops and now we do half-day workshops and sometimes they're just 90-minute workshops, so workshops aren't all the same."
"Is there a better way to show that we have done a lot of good work with these workshops?"
"How about counting the individual sessions rather than the full workshops?  In those 20+ workshops, we've addressed 115 distinct topics in sessions lasting anywhere between 30 minutes to 2 hours."
"Thanks.  That's useful."
"You're welcome.  We also have attendance data and ....."
"Thanks.  I think the number of topics is good enough for now."
"Well... it would be nice to know in advance what metrics are of interest to management.  I keep metrics that are relevant to me for the purpose of improving the workshops."

Lesson/Insight

  • When asked for metrics, understand the rationale for the request.  Ask questions (without sounding defensive or overprotective of data).
  • Avoid surprise requests for data by pro-actively engaging management in the determination of valuable metrics to be collected.
  • Don't just answer a question with data that you know is going to be misleading without further explanation, but do make an effort to be responsive and answer the question honestly. 

No comments: