Metrics Anecdote

Here is a short, very simple fictional (adapted from reality) anecdote about metrics in the realm of knowledge management.

"Barbara, how many workshops have we done in the past 8 years?"
"Let me check.... 25."
"We've done 25 workshops in the past 8 years?"
"Yes."

And the number 25, whatever it means, is now a data point for someone in management who needed to know how many workshops we have implemented.  Someone needed numbers.  To say "we conduct knowledge sharing workshops" isn't the same as "we have conducted 25 knowledge sharing workshops in the past 8 years."  And yet, that number is meaningless.  Here's why.

"Barbara, how many workshops have we done in the past 8 years?"
"Probably more than 20. Why do you ask?"
"I need to document our activities with some quantitative metrics."
"Oh, then the total number of workshops isn't that relevant or useful because we used to do full day workshops and now we do half-day workshops and sometimes they're just 90-minute workshops, so workshops aren't all the same."
"Is there a better way to show that we have done a lot of good work with these workshops?"
"How about counting the individual sessions rather than the full workshops?  In those 20+ workshops, we've addressed 115 distinct topics in sessions lasting anywhere between 30 minutes to 2 hours."
"Thanks.  That's useful."
"You're welcome.  We also have attendance data and ....."
"Thanks.  I think the number of topics is good enough for now."
"Well... it would be nice to know in advance what metrics are of interest to management.  I keep metrics that are relevant to me for the purpose of improving the workshops."

Lesson/Insight

• When asked for metrics, understand the rationale for the request.  Ask questions (without sounding defensive or overprotective of data).
• Avoid surprise requests for data by pro-actively engaging management in the determination of valuable metrics to be collected.
• Don't just answer a question with data that you know is going to be misleading without further explanation, but do make an effort to be responsive and answer the question honestly. 

Measuring Learning in the Context of Individual Performance Assessments

Abt Associates' Jacob Alex Klerman blogged about the need to proceed with caution when dealing with efforts to measure performance.  It can become a very deep and expensive rabbit hole.

I'd like to push the idea further from a knowledge management/organizational learning perspective.  There have been arguments for including "learning" metrics in individual performance assessments.  The (simplified) logic is that if you want to encourage t a particular behavior, you should measure it.

I'm not sure if there is research on this topic (I suspect there is), but my intuition tells me that while performance monitoring may be useful to identify and take action on under-performing employees, it is much less useful in rewarding high performance employees who are self-motivated in the first place.  You might force under-performing employees to comply with certain things by threatening them with bad performance assessments, but can you force an under-performing employee to learn more?  I doubt it.  Can you help a willing learner?  Yes, but adding a learning metric to their individual performance evaluation won't do it.

So, here's the question: How would/could a learning metric have a positive impact on employee learning?  Perhaps indirectly, by communicating the organization's recognition of learning as an important element of performance; by forcing conversations about what constitutes workplace learning, what is an effective learning strategy for individuals.

If the ultimate objective is to have these ongoing conversations about workplace learning and how it contributes to individual, team and organizational performance, then individual performance metrics may not be the most appropriate starting point.  They may be a minor component of a much broader strategy (see map below).

This also illustrates a point made by Beer, Finnstrom and Schrader (2016) in "The Great Traing Robbery," which is that training -- and leadership development in particular -- needs to be fully integrated with organizational development.

Two Key Elements of Personal Knowledge Management - Reflection and Experimentation

Clark Quinn blogged about "Experimentation and Reflection" earlier this week in the Modern Workplace Learning Magazine.  It's a recurring theme in my own reflections about Knowledge Management and Organizational Learning.

I like reading and interacting with people who work on similar issues yet from a slightly different perspective.  While most of the knowledge management and organizational learning professionals I know focus on the group or the organization as the relevant unit of analysis, Clark Quinn, Jane Hart and others who work in the arena of workplace learning within the broader umbrella of "learning and development" (L&D) have a much strong emphasis on the individual as the relevant unit of analysis.  I want to work at the intersection of KM/OL/ and L&D.

I'd like to argue that what's missing is an approach that bridges all three levels: Individual knowledge and learning, team knowledge and learning processes, and organizational knowledge and learning.

The foundation for robust knowledge management at the organizational level is a culture of learning. That culture is an aggregation of the behaviors and attitudes of all the individuals who work in that organization.  Therefore, it would make sense to suggest that knowledge management at the individual level (i.e, personal knowledge management) is an important foundation for knowledge management at the organizational level.

I have yet to see a knowledge management effort that pays attention to personal knowledge management and systematically links individual learning to team/group learning and organizational learning. [There is some research on the "microfoundations" of organizational learning which supports this idea.]

Putting some emphasis on the individual is not about rewarding individual incentives and addressing the "what's in it for me" attitude that we can encounter when pushing for more knowledge sharing.  Personal knowledge management, however, can have a significant impact on 1) personal motivation/job satisfaction; 2) engagement with peers/informal mentoring.

Quinn's post mentions the difficulty in measuring the informal learning progress that occurs through experimentation and reflection.

"I’m not sure that there are many tools that are expressly for tracking individual informal learning progress (though I’m using a new task/project management tool to create my todos and then mark them when done). Still, thinking consciously about learning goals and tracking progress could be a valuable adjunct to intentional learning." (Experimentation and Reflection, blog post of 2/9/2017)

I don't generally recommend bean counting when it comes to learning but here is an option to consider.

1. Document your informal learning activities (including your experimentation and reflection activities) in some fashion.  This is useful whether you want to measure progress or not.

2. Review your informal learning notes (however you have captured them) on a regular basis (weekly, monthly, whatever works for you).  Some of it will be observations.  some of it will be key insights, perhaps even some action items that you assigned yourself.   As you review notes from several months of informal learning activities, new insights will emerge, new actions to follow through with.  That's how the experimentation keeps evolving.  It's a form of agile learning.

3. Count key insights generated over time:  By the time you're done reviewing your notes, you know you've made progress.  If you need to report that progress to someone or you need a quantitative measure of progress, simply count the key insights you've generated over a month, keep track every month.  I don't think the goal is to keep increasing the number of insights.  Some are worth more than others. :)

If you track the number of miles you run (regardless of speed) for physical fitness purposes, you can track the number of key insights you generate out of your informal learning activities.   By itself, the process of tracking them down will generate new insights.

Another milestone - 100 students

Today, I reached 100 students in my Skillshare class and I'm still at 100% positive reviews.  I'm going to celebrate those two numbers and NOT go into what such wonderful metrics don't tell you.  There's always a story behind numbers.

____________________________________________________________

_____________________________________________________________